Ornith-1.5-35B-A3B-GGUF

GGUF quantizations of ornith-ai/Ornith-1.5-35B-A3B, made with an importance matrix, plus a KL-divergence comparison with the other published GGUFs of this model.

[!NOTE] These files were made for a quantization experiment and are evaluated only by the KLD measurements below, not by task benchmarks, long-context use or speed tests. Check them on your own workload before relying on them.

Files

The four "comparison only" files are there so that the effect of -hq in the tables below can be checked; at each size the -hq file is better. All files are Q4_K_M with up to two kinds of changes. The -hq part raises the tensors outside the routed experts to Q8_0: attention, the linear-attention (SSM) projections, the shared expert, and the MTP projection. The -exps part changes the routed expert tensors (ffn_{gate,up,down}_exps) in layers 0-39. The token embedding and the output keep the Q4_K_M types (Q4_K and Q6_K). All files keep the built-in MTP (NextN) layer, so they work with --spec-type draft-mtp. For image input, use the vision projector from the official repo (mmproj-Ornith-1.5-35B-BF16.gguf).

Quality

KL divergence (KLD) of the next-token distribution from BF16; lower is closer to the original. The other published GGUFs of this model are measured the same way. Rows marked "no -hq" are the comparison-only files in this repo.

• Each -hq file has a lower mean KLD, in both languages, than every published file of the same size or smaller. Q4_K_M-hq and IQ4_XS-exps-hq are also below every published Q4_K_M; the only published file below them is APEX I-Quality, 0.0011 lower than Q4_K_M-hq in Japanese at 1.3 GB more.
• Q8_0 for the non-expert tensors costs 0.6-0.7 GB and lowers mean KLD by 18-52%, as the "no -hq" rows show. Each routed expert runs for 8 of 256 slots, while the non-expert tensors run for every token, so their error weighs more per byte.
• In the 16-18 GB range, IQ3_S-exps-hq (16.7 GB) is 26% (ja) and 28% (en) below bartowski Q3_K_L (17.4 GB). IQ3_XXS-exps-down3-hq is another 1 GB smaller and still below Q3_K_L, with a lower tail than IQ3_S-exps-hq.
• Below 15 GB, IQ2_S-exps-down3-hq and IQ2_XS-exps-down3-hq are 16-42% below the files of similar size.
• Even Q8_0 is further from BF16 here than usual (mean KLD 0.013-0.020; BF16 perplexity is about 8.4 on both texts), so the absolute KLD numbers are higher than for many other models.
• Our imatrix and bartowski's give nearly the same result: IQ3_S-exps-hq made with bartowski's imatrix (Ornith-1.5-35B-A3B-imatrix.gguf, same rule file; not uploaded) measured 0.0949 / 0.1001, against 0.0927 / 0.1056 with ours. The gains above come from the mix, not the calibration data.

How these were made

• llama.cpp 9b56ccf (CPU build). The BF16 source is the official Ornith-1.5-35B-BF16.gguf, which includes the MTP layer.
• Imatrix: computed on Q8_0 over 367 chunks of 512 tokens from bartowski's calibration_datav3 and TFMC's ja/en C4 set. About a third of the chunks are wrapped in the model's chat format, including <think> and tool-call round trips, so that the chat and tool markers are part of the activations. The MTP layer is not run during imatrix collection.
• Mixed files: Q4_K_M plus the rules in tensor-types-<name>.txt, matched in order, first match wins. Q4_K_M uses no rule file. The expert rules cover layers 0-39 only, so the experts of the MTP layer (blk.40) keep the Q4_K_M types.

llama-quantize --imatrix imatrix.gguf --tensor-type-file tensor-types-IQ3_S-exps-hq.txt \
  Ornith-1.5-35B-BF16.gguf Ornith-1.5-35B-A3B-IQ3_S-exps-hq.gguf Q4_K_M

• KLD: llama-perplexity --kl-divergence against BF16 logits, context 1024, on Japanese Wikipedia (wiki40b-ja test, 16 chunks) and English Wikipedia (wikitext-2 test, 8 chunks). Neither text is in the calibration data. The other repos' files were downloaded in October 2026 and measured with the same build and settings.

License and credits

MIT License, the same as the original model.