Fanar-1-9B-Instruct — MLX 8-bit
An MLX build of QCRI/Fanar-1-9B-Instruct, quantised to 8 bits for Apple Silicon.
Upstream publishes safetensors only — no MLX build. This is it, converted with mlx_lm 0.32.0.
What the quantisation costs
An LLM quantisation is lossy by design, so there is no equivalence claim to make here — a cosine score against the original would be meaningless. What is worth reporting is the price, measured as perplexity against the unquantised original on the same held-out text:
| Build | Perplexity | vs bf16 |
|---|---|---|
| bf16 (unquantised) | 9.854 | — |
| 8-bit | 9.865 | +0.11% |
Eval set: 37712 characters of Arabic prose plus a short English passage,
written into the conversion script so it is fixed and auditable rather than sampled from a dataset.
Lower perplexity is better; a small positive delta is the normal price of 8-bit.
This is not a quality benchmark. Perplexity is not task accuracy, no Arabic benchmark was run, and upstream's own evaluation numbers are not reproduced here.
Use
pip install mlx-lm
python -m mlx_lm.generate --model ahmedandaloes/Fanar-1-9B-Instruct-MLX-8bit \
--prompt "ما هي عاصمة المملكة العربية السعودية؟" --max-tokens 200
What is in this repository
| File | |
|---|---|
*.safetensors, config.json, tokenizer* |
the MLX weights, as mlx_lm.convert wrote them |
README.md |
this card |
quantisation-cost.json |
the perplexity numbers above, machine-readable |
conversion.md |
the exact commands, and how to re-measure |
Size: 8.7 GiB.
Limits
- Only the 8-bit quantisation is here. Convert the original yourself for bf16.
- No chat-template or generation-quality testing beyond the one example above.
- No speed or memory measurement.
- Perplexity on one text is a weak signal; treat the delta as an order of magnitude, not a score.
Licence and attribution
Apache-2.0, as declared by upstream. A derivative work of QCRI/Fanar-1-9B-Instruct; credit for the model is theirs.