Ornith-1.5-9B / 1.0-9B — ASHQ1 Quantization (MTP-Aware)

ASHQ1 quantization of Ornith with explicit MTP (Multi-Token Prediction) handling — the whole MTP head (blk.32.* + nextn.*) is kept at Q8_0 and excluded from the classifier budget, while the priority queue optimises the main transformer layers by Σ(importance) × ΔMSE / MiB.

Note: File names contain "BF16" for HuggingFace parser compatibility — these are ASHQ1 quants, not BF16.

🧪 KLD-validated (battery by Soulfate24, ctx 512×64, Flash-Attention). PPL is a canary, KLD-vs-BF16 is the rank column. Honest placement: our 6500 is Nano-class fidelity at Quality-class size — Remix Quality-36 holds the rank here (KLD 0.0241 vs our 0.0763). PPL-only verdict lines retired per our own standing rule.

Quants — Ornith-1.5-9B (current)

File	Size	PPL (ctx 1024) / KLD	Description
`Ornith-1.5-9B-MTP-BF16-ASHQ1-6500.gguf`	6511 MiB	8.6541 / 0.0763 (solid)	Nano-class fidelity at Quality-class size; clean chunk-stability, no collapse
`Ornith-1.5-9B-MTP-BF16-TOPDOWN-6500.gguf`	6509 MiB	8.6337 / —	Experimental top-down mode, norms at Q4_K — same PPL (KLD pending)
`Ornith-1.5-9B-MTP-BF16-ASHQ1-4500.gguf`	4518 MiB	10.5057 / 0.3256 (degraded)	Compact; TOX beats it on both columns
`Ornith-1.5-9B-MTP-BF16-ASHQ1-4500-TOX.gguf`	4518 MiB	10.2194 / 0.2894 (degraded)	Better build of the pair (−0.286 PPL / −0.036 KLD); sub-4 buys real fidelity, just not above 0.20
`Ornith-1.5-9B-i1-Q6_K.gguf`	7209 MiB	9.2025 / —	Uniform Q6_K with imatrix (reference)

Benchmark vs ASHQ1-Remix (same model; PPL = canary, KLD = rank)

Method	Size	PPL	KLD	Verdict
ASHQ1 (this repo)	6511 MiB	8.6541	0.0763	Nano-class fidelity at Quality-class size
ASHQ1 top-down (this repo)	6509 MiB	8.6337	—	same PPL (KLD pending)
Remix Quality-36pc	6388 MiB	8.9895	0.0241	👑 rank holder: 3.2× KLD at 123 MiB smaller
Remix Q5_K_M-imx (stock)	6335 MiB	8.6345	0.0665	stock twin in the same compression zone

Our 6500 takes the PPL column; Remix Quality takes the rank column. Both truths stay on the card.

Top-down experiment

--top-down reverses the greedy: every quantizable tensor starts at F16 — norms included — and is downgraded cheapest-loss-first until the budget fits. Result on 1.5 @6500: all 177 norm tensors sank to the Q4_K floor, zero tensors left at F16 — and PPL didn't move (8.6337 vs 8.6341, σ=0.06). Takeaway: norms on this architecture are free real estate — quantizing all of them to Q4_K costs nothing, but PPL is too coarse a metric to separate the two allocation paths. Sensitive tasks (HumanEval, MMLU) may tell them apart.

How ASHQ1 Works (v7)

Base: norms/ssm_params → F16, output/token_embd pinned → Q5_K (outside budget — llama.cpp writes them at the output/token types unconditionally), rest → Q4_K (--allow-q3-or-lower: CAN_Q3 types → IQ2_XXS).
MTP head (whole blk.32 layer, detected by excluding it from n_layers) → Q8_0, outside budget.
Priority queue over tied groups + singletons: Σ(importance) × ΔMSE / MiB, theoretical MSE metric (ΔMSE = 2^{−2·bpw₁} − 2^{−2·bpw₂}, K-quants +0.1 effective bpw). Zero-cost steps (IQ4_NL→Q4_K) apply as free upgrades inline.
Drain best-first until the target fits. Estimator accuracy: ±3 MiB vs llama-quantize --dry-run.

Method, source code and older benchmarks: wepiqx/ASHQ1.

Usage

llama.cpp

# Chat mode with built-in Jinja template (recommended)
llama-cli \
  -m Ornith-1.5-9B-MTP-BF16-ASHQ1-6500.gguf \
  --jinja \
  -ngl 99 \
  -c 8192

Recommended sampling: temperature 0.6–1.0, top_p 0.95, top_k 20.

Ollama

The model's built-in Jinja template is auto-detected by Ollama — no need to specify TEMPLATE manually.

Create a Modelfile:

FROM ./Ornith-1.5-9B-MTP-BF16-ASHQ1-6500.gguf

PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95

Build and run:

ollama create ornith-15-ashq1-6500 -f Modelfile
ollama run ornith-15-ashq1-6500

LM Studio

Open LM Studio
Drag the desired GGUF file into the app
Set GPU Offload to 99 layers
Set context length to 8192+
Set preset sampling: temperature 0.6, top_p 0.95, top_k 20
Start chatting

Quantization Config — 1.5 @6500

Generated by ASHQ1 (v7, current code). Reproduce with: python main.py --model Ornith-1.5-9B-MTP-BF16.gguf --imatrix Ornith-1.5-9B-imatrix.gguf --size 6500 --run

llama-quantize arguments (click to expand)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.ffn_down=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.ffn_gate=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.ffn_up=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.post_attention_norm=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.attn_q_norm=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.attn_output=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.attn_norm=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.attn_k_norm=Q8_0"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.(3|7|23|27|(?:31|32))\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|23|27|(?:31|32))\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|23|27|(?:31|32))\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.((?:17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.(27|31)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(28|30)\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.((?:17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(11|15|19)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.(11|15|19)\.attn_q=Q6_K"
--tensor-type "(blk|BLK)\.(11|15|19)\.attn_v=Q6_K"
--tensor-type "(blk|BLK)\.((?:26|27|28|29|30))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25))\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13|14|15|16))\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13|14|15|16))\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.(22|(?:24|25|26)|29)\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.(19|23)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15)\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_gate=Q4_K"
--tensor-type ".*output_norm.*=F16"

Quants — Ornith-1.0-9B (previous generation)

File	Size	Description
`Ornith-1.0-9B-MTP-BF16-ASHQ1-4850.gguf`	4.7 GB	Tiny, IQ tiers (`--allow-q3-or-lower`)
`Ornith-1.0-9B-MTP-BF16-ASHQ1-5500.gguf`	5.4 GB	Compact
`Ornith-1.0-9B-MTP-BF16-ASHQ1-6500.gguf`	6.4 GB	Balanced
`Ornith-1.0-9B-MTP-BF16-ASHQ1-7300.gguf`	7.2 GB	Best

Built with an older ASHQ1 revision (only nextn.* pinned as MTP, output/embd at IQ4_NL). The v7 configs above are not bit-identical to these files. Non-MTP 1.0 quants (deepreinforce-ai_Ornith-1.0-9B-BF16-ASHQ1-*) in commit history.

Hardware Requirements

Variant	Size (GGUF)	VRAM (ctx 8192)	Minimum GPU
4500	4.4 GB	~5.5 GB	8 GB
6500	6.4 GB	~7.5 GB	8 GB

Notes

Source BF16 (1.5): MTP-ready BF16 with the head merged in (blk.32 + nextn.*)
Imatrix: custom Ornith-1.5 imatrix (248 entries)
Architecture: Qwen3.5-9B (32 layers, GQA, SSM, MTP, 262k context)
MTP heads deployed at Q8_0, output/token_embd at Q5_K
Quantization and tuning by wepiqx
Built with llama.cpp
License: MIT

wepiqx/Ornith-1.5-9B-MTP-ASHQ1-GGUF