ALLaM-7B-Instruct — MLX 8-bit

An MLX build of humain-ai/ALLaM-7B-Instruct-preview, Saudi Arabia's Arabic instruct model (SDAIA), quantised to 8 bits for Apple Silicon.

Upstream publishes safetensors only — no MLX build. This is it, converted with mlx_lm 0.32.0.

What the quantisation costs

An LLM quantisation is lossy by design, so there is no equivalence claim to make here — a cosine score against the original would be meaningless. What is worth reporting is the price, measured as perplexity against the unquantised original on the same held-out text:

Build Perplexity vs bf16
bf16 (unquantised) 17.449 —
8-bit 17.445 -0.02%

Eval set: 2360 characters of Arabic prose plus a short English passage, written into the conversion script so it is fixed and auditable rather than sampled from a dataset. Lower perplexity is better; a small positive delta is the normal price of 8-bit.

This is not a quality benchmark. Perplexity is not task accuracy, no Arabic benchmark was run, and upstream's own evaluation numbers are not reproduced here.

Use

pip install mlx-lm
python -m mlx_lm.generate --model ahmedandaloes/ALLaM-7B-Instruct-MLX-8bit \
    --prompt "ما هي عاصمة المملكة العربية السعودية؟" --max-tokens 200

What is in this repository

File
*.safetensors, config.json, tokenizer* the MLX weights, as mlx_lm.convert wrote them
README.md this card
quantisation-cost.json the perplexity numbers above, machine-readable
conversion.md the exact commands, and how to re-measure

Size: 6.9 GiB.

Limits

  • Only the 8-bit quantisation is here. Convert the original yourself for bf16.
  • No chat-template or generation-quality testing beyond the one example above.
  • No speed or memory measurement.
  • Perplexity on one text is a weak signal; treat the delta as an order of magnitude, not a score.

Licence and attribution

Apache-2.0, as declared by upstream. A derivative work of humain-ai/ALLaM-7B-Instruct-preview; the model is SDAIA's work.