Ornith-1.5-9B-SLQ-4.5bpw (MTP-grafted)

SLQ (Statistically-Lossless Quantization) packed checkpoint of ornith-ai/Ornith-1.5-9B, at 4.50 bpw via MLX-LM's QuantizedLinear. The MTP (Multi-Token Prediction) head weights have been grafted from the BF16 base model to preserve prediction fidelity.

Quick start

pip install mlx mlx-lm
python -m mlx_lm.generate --model hermitdave/Ornith-1.5-9B-MLX-SLQ-4.5bpw \
  --prompt "The capital of France is" --max-tokens 64

Requires MLX ≥ 0.27 (Apple Silicon only).

This checkpoint uses a custom architecture file (slq_model.py) that wraps the base mlx_lm.models.qwen3_5 model with per-layer QuantizedLinear layers. Loading it requires trust_remote_code=True.

Model details

Parameter	Value
Base model	ornith-ai/Ornith-1.5-9B
Architecture	Qwen3_5 hybrid (linear_attention + full_attention)
Parameters	~9.0B
Quantization	per-layer non-uniform affine (SLQ), group_size=64
Target bpw	4.50 → measured 5.00 bpw
Format	mlx-packed-affine

Bitwidth distribution

Bitwidth	Tensor count
8-bit	25
6-bit	43
4-bit	180

Size

|| Artifact | Size | vs BF16 | ||----------|------|---------| || BF16 base | ~23.0 GiB | — | || SLQ 4.5bpw (MTP-grafted) | ~9.2 GiB | ~2.5× smaller |

MTP Graft

The MTP (Multi-Token Prediction) head weights are kept in BF16 precision to preserve next-token prediction fidelity. These weights were extracted from the BF16 base model and grafted onto the SLQ quantized checkpoint as a dedicated safetensors shard (model-graft.safetensors). This adds ~487 MB to the quantized artifact while maintaining the SLQ backbone's compression.

Fidelity (measured)

Calibration: FineWeb-Edu, 131,072 tokens (32×1024). Allocator: per-layer greedy, EAR target 0.99.

Metric	Value
PPL ratio (FineWeb-Edu 512-seq)	1.055
Symmetric KLD (FineWeb-Edu 512-seq)	0.405
Zero-shot mean recovery (4 tasks, n=400)	0.9685
Task-lossless verdict	No

Per-task recovery (capped at 400 examples/task)

Task	BF16	SLQ 4.5bpw	recovery	95% CI
arc_challenge	0.5200	0.4950	0.9519	[-0.0568, +0.0068]
arc_easy	0.7775	0.7825	1.0064	[-0.0190, +0.0290]
hellaswag	0.5000	0.4925	0.9850	[-0.0252, +0.0102]
winogrande	0.5400	0.5025	0.9306	[-0.0770, +0.0020]

Note: the full benchmark sets are ~15.5k examples/task; this run uses a 400-example cap for speed.

Performance

Model	TPS
BF16	21.3
SLQ 4.5bpw	20.6

Original model card

The original model card is at ornith-ai/Ornith-1.5-9B.

Methodology

Built with slq-mlx. Calibration: FineWeb-Edu, 128 samples × 2048 tokens. Allocator: per-layer greedy.

Attribution

Quantization tool: slq-mlx (Hermes Agent workflow)
Base model: ornith-ai/Ornith-1.5-9B
MTP graft: BF16 MTP head weights extracted from base model and grafted onto SLQ checkpoint
Calibration data: FineWeb-Edu (HuggingFaceFW/fineweb-edu)
Benchmark harness: custom zero-shot scorer (slq-mlx scripts/run_zero_shot.py), paired BF16 vs SLQ

hermitdave/Ornith-1.5-9B-MLX-SLQ-4.5bpw