Ornith-1.5-9B-SLQ-4.5bpw (MTP-grafted)
SLQ (Statistically-Lossless Quantization) packed checkpoint of ornith-ai/Ornith-1.5-9B, at 4.50 bpw via MLX-LM's QuantizedLinear. The MTP (Multi-Token Prediction) head weights have been grafted from the BF16 base model to preserve prediction fidelity.
Quick start
pip install mlx mlx-lm
python -m mlx_lm.generate --model hermitdave/Ornith-1.5-9B-MLX-SLQ-4.5bpw \
--prompt "The capital of France is" --max-tokens 64
Requires MLX ≥ 0.27 (Apple Silicon only).
This checkpoint uses a custom architecture file (slq_model.py) that wraps the base mlx_lm.models.qwen3_5 model with per-layer QuantizedLinear layers. Loading it requires trust_remote_code=True.
Model details
| Parameter | Value |
|---|---|
| Base model | ornith-ai/Ornith-1.5-9B |
| Architecture | Qwen3_5 hybrid (linear_attention + full_attention) |
| Parameters | ~9.0B |
| Quantization | per-layer non-uniform affine (SLQ), group_size=64 |
| Target bpw | 4.50 → measured 5.00 bpw |
| Format | mlx-packed-affine |
Bitwidth distribution
| Bitwidth | Tensor count |
|---|---|
| 8-bit | 25 |
| 6-bit | 43 |
| 4-bit | 180 |
Size
|| Artifact | Size | vs BF16 | ||----------|------|---------| || BF16 base | ~23.0 GiB | — | || SLQ 4.5bpw (MTP-grafted) | ~9.2 GiB | ~2.5× smaller |
MTP Graft
The MTP (Multi-Token Prediction) head weights are kept in BF16 precision to preserve next-token prediction fidelity. These weights were extracted from the BF16 base model and grafted onto the SLQ quantized checkpoint as a dedicated safetensors shard (model-graft.safetensors). This adds ~487 MB to the quantized artifact while maintaining the SLQ backbone's compression.
Fidelity (measured)
Calibration: FineWeb-Edu, 131,072 tokens (32×1024). Allocator: per-layer greedy, EAR target 0.99.
| Metric | Value |
|---|---|
| PPL ratio (FineWeb-Edu 512-seq) | 1.055 |
| Symmetric KLD (FineWeb-Edu 512-seq) | 0.405 |
| Zero-shot mean recovery (4 tasks, n=400) | 0.9685 |
| Task-lossless verdict | No |
Per-task recovery (capped at 400 examples/task)
| Task | BF16 | SLQ 4.5bpw | recovery | 95% CI |
|---|---|---|---|---|
| arc_challenge | 0.5200 | 0.4950 | 0.9519 | [-0.0568, +0.0068] |
| arc_easy | 0.7775 | 0.7825 | 1.0064 | [-0.0190, +0.0290] |
| hellaswag | 0.5000 | 0.4925 | 0.9850 | [-0.0252, +0.0102] |
| winogrande | 0.5400 | 0.5025 | 0.9306 | [-0.0770, +0.0020] |
Note: the full benchmark sets are ~15.5k examples/task; this run uses a 400-example cap for speed.
Performance
| Model | TPS |
|---|---|
| BF16 | 21.3 |
| SLQ 4.5bpw | 20.6 |
Original model card
The original model card is at ornith-ai/Ornith-1.5-9B.
Methodology
Built with slq-mlx. Calibration: FineWeb-Edu, 128 samples × 2048 tokens. Allocator: per-layer greedy.
Attribution
- Quantization tool: slq-mlx (Hermes Agent workflow)
- Base model: ornith-ai/Ornith-1.5-9B
- MTP graft: BF16 MTP head weights extracted from base model and grafted onto SLQ checkpoint
- Calibration data: FineWeb-Edu (HuggingFaceFW/fineweb-edu)
- Benchmark harness: custom zero-shot scorer (
slq-mlxscripts/run_zero_shot.py), paired BF16 vs SLQ