Ornith-1.5-122B — MLX imatrix-allocated mixed quant + MTP

An MLX build of Ornith-1.5-122B that fits a 64 GB Mac with room to spare, with the MTP head grafted for speculative decoding. The bit allocation is driven by the publisher's importance matrix rather than chosen by hand, and every number below was measured on an M3 Max (64 GB) rather than copied from a model card.

47.37 GB on disk (45.95 GB text + 1.42 GB MTP) · 23 shards · runs on stock mlx_lm and on oMLX.

This is the smaller, faster sibling of Ornith-1.5-122B-MLX-Mixed2-3-8bit-MTP. Read Which one should I download before picking — they are not equivalent, and the difference is measured below.

Which one should I download

Take this build if you want the fastest option that still fits comfortably, and you accept a measurable distributional drift from the previous build.

Take the previous build if you want maximum fidelity to the source weights. It is 1.65 GB larger and ~16% slower, but this build's drift is real — perplexity cannot see it, top-1 and KL divergence can.

Bit allocation

Chosen by measuring each tensor's expected output error at each candidate bit width,

mse(bits) = Σ_c  E[A_c²] · ‖ W[:,c] − Q_bits(W)[:,c] ‖²

where E[A_c²] is the per-input-channel activation energy from the publisher's importance matrix (in_sum2 / counts, 9,413 calibration chunks). That is the expected squared error each tensor contributes to its own output, so tensors can be ranked against each other, and a size budget can be spent greedily on MSE-reduction-per-byte.

Where the metric was overruled

mse(bits) measures a tensor's immediate output error and is blind to downstream amplification. It wanted two things that both reference recipes contradict:

• ssm_out at 2-bit. Its measured MSE is tiny (0.0098 vs attn_qkv's 391.5, 40,000× smaller) because its input activations are small. But unsloth use Q5_K and AtomicChat use 8-bit. It feeds the residual stream, so small absolute error can still matter.
• ffn_down_exps at 2-bit, with gate/up at 3-bit. Both references give down more bits than up — AtomicChat's AD-<ffn_down>-<ffn_up> naming encodes this, and unsloth do the same at every layer. Its output lands directly in the residual stream.

In both cases the reference recipes won. Where the greedy agreed with the references — the gate/up depth ramp, the router at 8-bit — its answer was kept, since that is where the metric adds information rather than contradicting it.

The honest summary: the imatrix's real contribution here was narrow. The depth ramp was genuine corroboration between two independent methods, and the router choice is one I would have got wrong by hand. But every byte of the 1.65 GB saving came from the non-expert group — which every reference already said to compress. The metric mostly served as a check, not a discovery, and its two headline suggestions were both rejected.

Measured quality

Reference is the previous build (47.6 GB text, uniform 2-bit gate/up + 3-bit down + 8-bit rest), scored on 4,096 positions of wikitext-2 test with identical tokenisation:

Read this before quoting those numbers

Perplexity cannot see the difference; top-1 and KLD can. If we had stopped at PPL we would have reported "no quality loss", and that would have been wrong. A paired t-test gives t = 1.02 — well inside noise — while top-1 agreement is 80.22% and KL divergence is 0.243. PPL is the least sensitive of the three.

The KLD baseline is not BF16. Q8_0 is 132 GB and cannot run on a 64 GB machine, so the reference is our own previous build, not the original weights. The previous build is itself roughly 0.13 from BF16 (AtomicChat measure 0.13030 for their comparable 47.6 GB rung). If the two builds' errors were independent, this build would be ~0.37 from BF16 — worse than unsloth's UD-Q2_K_XL at 0.195. If they are correlated, as two quantizations of the same source should be, it is less. This measurement cannot distinguish those cases, and we are not going to imply otherwise.

The reassuring part: 42.8% of disagreements occur where the previous build itself had a margin below 0.5, and 10.4% below 0.1 — near-ties where either token is defensible.

For context, published figures on this model:

MTP speculative decoding

The MTP head (blk.48.*, 2.5237 B params) is converted to 4-bit and grafted as an extra shard with keys prefixed language_model.mtp.*.

Measured through oMLX, 300-token generations, temperature 0, one model reload per setting, warm-up discarded:

Depth 1 is optimal here, unlike the previous build where depth 2 won. First-draft acceptance is actually better (67.4% vs 65.6%) — the head still predicts well off the reallocated base — but second-draft acceptance fell (58.2% vs 62.8%), so chaining a weaker second draft no longer pays for the extra verify.

The mechanism is verify cost, not draft quality. Cycles/s drops from 47.7 for single-token decode to 31.8 for a two-token verify — a 1.5× penalty, versus 1.26× on the previous build. The faster backbone makes each speculative cycle relatively more expensive, which is why the relative gain is smaller (1.13× vs 1.40×) even though absolute throughput is higher.

Note 53.7 vs 53.3 is 0.75% — inside noise. Depth 1 and 2 are effectively tied; depth 1 is recommended because it is simpler and has the better first-draft acceptance.

oMLX setup note: the MTP keys must be listed in model.safetensors.index.json's weight_map. oMLX's _checkpoint_weight_prefix scans only the index and returns None otherwise, which silently disables Lightning MTP with no error. config.json additionally needs mtp_num_hidden_layers and text_config.mtp_num_hidden_layers. Both are already set here.

How it was built

Source: AtomicChat/Ornith-1.5-122B-GGUF Q8_0 shards (affine, same scale+offset structure as MLX affine, so the port is faithful). Importance matrix: AtomicChat/Ornith-1.5-122B-GGUF-metrics.

The pipeline streams one tensor at a time — peak RAM is a single layer, not the model — and applies the GGUF→HF/MLX transforms that llama.cpp's gated-delta-net layout requires. These are not optional: without them the model loads cleanly, passes every shape check, and generates loops. The value-head grid permutation (perm(i) = 4*(i % 16) + (i // 16)) propagates into ssm_a, dt_bias, the in_proj_* rows, out_proj columns, and the conv1d channels.

Limitations

• Text only. Vision weights are excluded.
• The drift caveat above. The KLD baseline is our previous build, not BF16.
• Measured on one machine (M3 Max, 64 GB). Other hardware will differ.
• The 2-bit regime costs real quality. On this model the published KL curve is a cliff, not a slope: ~0.195 at 44.5 GB, ~0.10 at 60 GB, ~0.035 at 80 GB. Anything that fits a 64 GB Mac is in the expensive part of that curve.
• Perplexity figures are from a 4,096-token sample, not the ~100k+ used for published numbers. Absolute PPL is therefore noisy; only the ratio against the same text is meaningful.

Credits

• ornith-ai/Ornith-1.5-122B — the original weights.
• Qwen — Ornith-1.5-122B is built on Qwen 3.5 122B (e.g. Qwen/Qwen3.5-122B-A10B), which is licensed Apache 2.0. That licence governs the base weights this build derives from.
• AtomicChat — the Q8_0 GGUF source and the importance matrix, plus the published KL/top-1 ladder used here as the reference for the bit-allocation floors.
• unsloth — their UD-Q2_K_XL GGUF's per-tensor type table was read directly and used as a second independent reference recipe.
• Hermes Agent — conversion, imatrix-driven allocation, MTP graft, and all measurements.

License

Apache 2.0, following Qwen 3.5, the base model Ornith-1.5-122B is built on.

This repository is a quantization of ornith-ai/Ornith-1.5-122B, which derives from Qwen 3.5 122B (Apache 2.0). No weights were retrained, fine-tuned, or otherwise modified in value beyond lossy numeric quantization; the architectural and MTP changes are format-level conversions for the MLX runtime, described in full above.

Note: the upstream ornith-ai/Ornith-1.5-122B model card declares license: mit, but its LICENSE file is absent (the URL 404s). We follow the Apache 2.0 terms of the Qwen 3.5 lineage it is built on, which is the more conservative and better-documented choice.

As required by Apache 2.0 §4, attribution notices for the upstream work are retained in the Credits section above, and all changes made to produce this build are documented in this card — specifically: GGUF→MLX affine requantization at mixed bit widths, the gated-delta-net parameter transforms required by llama.cpp's layout, and the grafting of the MTP head as an additional shard.