A 4-bit MLX quantization of sabicap/sabi-noema (Qwen3.8-27B further trained with reinforcement learning for agents, code and tool use) made with GPTQ instead of plain round-to-nearest.
MLX affine 4-bit, group size 64, bf16 scales, 15 GB: the same layout as a standard mlx_vlm.convert 4-bit checkpoint, so any tool that reads MLX 4-bit (mlx-lm, mlx-vlm) loads it unchanged. Only the 64 text decoder layers were re-quantized; the vision tower, embeddings and output head are copied unchanged from the round-to-nearest 4-bit conversion.
Same method as nicopingu/Qwen3.8-27B-MLX-4bit-GPTQ.
KL divergence against an 8-bit MLX conversion of Sabi-Noema over the full vocabulary, mean over 8,192 tokens per text, and the share of tokens whose top-1 prediction matches the reference. Lower KL is better. "RTN 4-bit" is a plain mlx_vlm.convert 4-bit conversion (group size 64).
| Text | RTN 4-bit | This model (GPTQ) | Top-1 match RTN / GPTQ |
|---|---|---|---|
| Technical docs and Python code (not related to the calibration set) | 0.0521 | 0.0347 (-33 %) | 91.3 % / 92.6 % |
| English, wikitext-2 test | 0.0394 | 0.0261 (-34 %) | 90.2 % / 92.5 % |
| French, Wikipedia articles | 0.0299 | 0.0196 (-35 %) | 92.0 % / 93.3 % |
| Code, GitHub files (repositories not in the calibration set) | 0.0285 | 0.0206 (-28 %) | 95.7 % / 96.4 % |
| Function calling dialogues (glaive format) | 1.079 | 0.550 (-49 %) | 78.5 % / 87.2 % |
In plain terms: the share of tokens that differ from the 8-bit model drops by about 40 % on tool calls (21.5 % to 12.8 %) and by 15 to 23 % on text and code.
Agent test under PLI (24 real agent turns, DFlash2 draft, Mac Studio M1 Ultra): 24/24 valid tool calls for both models, same prompt reading speed (249 vs 248 tok/s), same decoding speed on the turns where both models write the same answer, and a higher draft acceptance on those turns (86 % vs 82 %). On the other turns the GPTQ model sometimes chooses different, longer actions, so end-to-end speed depends on the task.
Caveats, stated plainly:
mlx_lm/quant/gptq.py.pip install -U mlx-vlm
python -m mlx_vlm.generate --model nicopingu/Sabi-Noema-MLX-4bit-GPTQ --max-tokens 200 --prompt "Write a Python function that parses an ISO 8601 date."
Text-only with mlx-lm:
from mlx_lm import load, generate
model, tokenizer = load("nicopingu/Sabi-Noema-MLX-4bit-GPTQ")
print(generate(model, tokenizer, prompt="Hello", max_tokens=100))
I run this model every day with PLI (Apache-2.0), an OpenAI- and Anthropic-compatible inference server for Apple Silicon, tuned for the M1 Ultra (fork of TensorFold), with the z-lab/Qwen3.8-27B-DFlash2 draft.
Apache-2.0, as sabicap/sabi-noema and its base Qwen/Qwen3.8-27B. This repository only changes the quantization of the weights; all credit for the model goes to sabicap and the Qwen team.
model-00003-of-00003.safetensors 5.4 GBmodel-00002-of-00003.safetensors 5.4 GBmodel-00001-of-00003.safetensors 5.3 GBsabicap/sabi-noema