GLM 5.1 optimized to run comfortably on a Mac Studio M3 512. This is the smaller, compact version. Alternatives: balanced, quality-first
- A mixed-precision quant that balances speed, memory, and accuracy.
- 3-bit baseline with important layers at 4, 8 and BF16.
- Fits into ~280 GB memory, leaving plenty of room to run parallel models (ex: Minimax M2.7, Qwen 3.6 35B).
Usage
# Start server at http://localhost:8080/chat/completions
uvx --from mlx-lm mlx_lm.server \
--host 127.0.0.1 \
--port 8080 \
--model spicyneuron/GLM-5.1-MLX-2.9bit
Benchmarks
| metric | baa-ai/GLM-5.1-RAM-270GB-MLX | 2.9 bit (this model) | 3.6 bit | 4.5 bit |
|---|---|---|---|---|
| bpw | 3.110 | 2.906 | 3.645 | 4.538 |
| base memory | 269.303 | 251.702 | 315.648 | 392.992 |
| peak memory (1024/512) | 291.257 | 272.358 | 341.020 | 424.067 |
| prompt tok/s (1024) | 194.958 ± 0.075 | 194.216 ± 0.167 | 190.508 ± 0.880 | 193.563 ± 0.094 |
| gen tok/s (512) | 21.381 ± 0.050 | 19.527 ± 0.035 | 17.873 ± 0.156 | 17.259 ± 0.032 |
| kl mean* | 0.686 ± 0.054 | 0.268 ± 0.009 | 0.117 ± 0.004 | 0.048 ± 0.002 |
| kl p95* | 1.478 ± 0.054 | 0.537 ± 0.009 | 0.236 ± 0.004 | 0.097 ± 0.002 |
| perplexity | 4.780 ± 0.020 | 4.118 ± 0.016 | 3.945 ± 0.016 | 3.920 ± 0.016 |
| piqa | 0.776 ± 0.010 | 0.794 ± 0.009 | 0.820 ± 0.017 | 0.814 ± 0.017 |
* GLM 5.1 KL divergence calculated against the largest quant I could run locally (~495 GB), so real KL is higher.
Tested on a Mac Studio M3 Ultra with:
mlx_lm.kld --baseline-model path/to/mlx-full-precision
mlx_lm.perplexity --sequence-length 2048 --seed 123
mlx_lm.benchmark --prompt-tokens 1024 --generation-tokens 512 --num-trials 5
mlx_lm.evaluate --tasks piqa --seed 123 --num-shots 0 --limit 500
mlx_lm.kld is approximate, based on top_k not full logits. Here's the code.
Methodology
Quantized with a mlx-lm fork, drawing inspiration from Unsloth/AesSedai/ubergarm style mixed-precision GGUFs. MLX quantization options differ from llama.cpp, but the principles are the same:
- Sensitive layers like MoE routing, attention, and output embeddings get higher precision
- More tolerant layers like MoE experts get lower precision