Ornith 1.5 122B for mlx-serve (mixed 4/8-bit)
An unofficial Apple-silicon MLX mixed-precision quantization of ornith-ai/Ornith-1.5-122B, tuned to retain quality where it matters while reducing deployment memory.
Pack size: 67 GiB. Recommended unified memory: 128 GB or more for practical serving headroom.
Quantization
Affine group-size-64 quantization for mlx-serve: dense MLP projections use 4-bit weights; attention projections, embeddings, LM head, routing-sensitive and selected shared-expert projections use 8-bit weights. Norms, scales, and biases remain BF16.
Quickstart
Load the pack with mlx-serve:
mlx-serve serve --model pgmharikrishnan/Ornith-1.5-122B-MLX-Serve-mixed-4-8bit
Then use its OpenAI-compatible API:
curl http://127.0.0.1:11234/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"pgmharikrishnan/Ornith-1.5-122B-MLX-Serve-mixed-4-8bit","messages":[{"role":"user","content":"Explain why mixed precision helps model quantization."}]}'
Results
Teacher-forced fidelity
Compared against the BF16 base model on 256 tokens of fidelity_text.txt, scoring each next-token argmax across the full 248,320-token vocabulary. Candidate weights were dequantized one layer at a time on CPU.
| Pack | Full-vocabulary top-1 agreement | 85% gate |
|---|---|---|
| This mixed 4/8-bit pack | 88.28% | PASS |
Official ornith-ai/Ornith-1.5-122B-MLX-4bit |
84.77% | FAIL |
The mixed pack passes the release criterion (at least 85% top-1 and no more than 2 percentage points below the official pack). The official 4-bit pack scores 84.77%, below the 85% threshold.
Standard-v2 comparison
Measured with mlx-serve 26.10.2 on Apple M4 Max, 2026-10-23. Decode rates are generated tokens per second; “best” is the benchmark’s best decode mode.
| Pack | Size | MMLU-Pro (400) | GSM8K (50) | Coding (6) | Plain decode | Best decode | Prefill |
|---|---|---|---|---|---|---|---|
| This mixed 4/8-bit pack | 66.8 GiB | 347/400 (86.75%) | 48/50 (96%) | 6/6 | 52.55 tok/s | 52.27 tok/s | 98.38 tok/s |
| Official 4-bit pack | 64.0 GiB | 350/400 (87.5%) | 47/50 (94%) | 6/6 | 73.59 tok/s | 73.53 tok/s | 219.63 tok/s |
Both packs used the same standard-v2 suite and seed. The mixed pack uses about 2.8 GiB more disk space and has lower decode and prefill throughput in this run.
Attribution
Ornith 1.5 was created by ornith-ai. This independent quantization preserves the upstream MIT license; see the base model for its model card and terms.