Nanbeige4.3-33B-MLX-4bit
Nanbeige/Nanbeige4.3-33B quantized to MLX 4-bit for Apple Silicon, with speed and quality numbers measured on a 64GB Mac mini (M5 Pro).
All credit for the model goes to the Nanbeige team. This repo only contains a quantized copy. Original license (Apache 2.0) applies.
How to run
pip install mlx-lm
mlx_lm.generate --model pratikgx/Nanbeige4.3-33B-MLX-4bit --prompt "Explain the Gettysburg Address in three sentences." --max-tokens 1024
Or chat with it:
mlx_lm.chat --model pratikgx/Nanbeige4.3-33B-MLX-4bit
Note: the chat template opens a <think> block, so the model reasons first and then answers. Give it enough --max-tokens for the thinking plus the answer.
Quantization details
| Format | MLX affine quantization |
| Bits | 4 (group size 64, ~4.5 bits per weight incl. scales/biases) |
| Size on disk | 18.6 GB (18,630,506,496 bytes of weights) |
| Quantized layers | all linear layers, including embeddings and lm_head |
| Source | bf16 weights of Nanbeige/Nanbeige4.3-33B |
| Tooling | mlx 0.32.4, mlx-lm 0.32.2 (mlx.core.quantize, same math as mlx_lm.convert -q) |
Measured on a Mac mini M5 Pro 64GB
Setup: Apple M5 Pro, 64 GB unified memory, macOS 27.0.1, on AC power, mlx 0.32.4 / mlx-lm 0.32.2, greedy decoding (temp 0). No thermal or performance warnings were recorded. Benchmarked October 25, 2026.
Speed
256 generated tokens per test. 1 warmup, then 3 runs; the table shows the median run.
| Context (prompt tokens) | Thinking | Prompt processing (tok/s) | Generation (tok/s) | Time to first token | Peak memory |
|---|---|---|---|---|---|
| 512 | on | 310.8 | 14.38 | 1.65 s | 19.32 GB |
| 512 | off | 358.2 | 14.51 | 1.43 s | 19.32 GB |
| 4,096 | on | 350.4 | 13.85 | 11.70 s | 20.21 GB |
| 4,096 | off | 355.8 | 13.75 | 11.52 s | 20.21 GB |
| 16,384 | on | 298.2 | 11.69 | 54.97 s | 22.82 GB |
| 32,768 | on | 252.0 | 9.93 | 130.10 s | 26.37 GB |
16k and 32k were only measured with thinking on.
Quality (small sanity checks, thinking on)
| Test | Score |
|---|---|
| Math word problems | 20/20 |
| Coding tasks | 10/10 |
| Instruction following | 10/10 |
Needle in a haystack (thinking on)
| Context | Needle depth | Found | Prompt processing (tok/s) | Peak memory |
|---|---|---|---|---|
| 32k | 10% | yes | 256.9 | 26.32 GB |
| 32k | 30% | yes | 260.5 | 26.35 GB |
| 32k | 50% | yes | 261.3 | 26.35 GB |
| 32k | 70% | yes | 253.7 | 26.35 GB |
| 32k | 90% | yes | 259.0 | 26.35 GB |
| 64k | 25% | yes | 195.4 | 33.44 GB |
| 64k | 50% | yes | 200.0 | 33.44 GB |
| 64k | 75% | yes | 200.6 | 33.44 GB |
| 128k | 50% | yes | 136.4 | 47.75 GB |
The 128k test (131,008 prompt tokens) ran on the 64GB machine without running out of memory and took about 16 minutes end to end.
Caveats
- One machine, one session. Speed numbers are medians of 3 runs; quality and needle tests were run once.
- The quality sets are small (20/10/10 items) and were written for this benchmark, so treat them as sanity checks, not a leaderboard score.
- Everything was measured with my own harness on top of
mlx_lm, not an official evaluation suite. - The benchmarked copy was converted on the Mac (Metal) from the same bf16 weights with the same settings as this upload (converted on CPU). Tensor names, shapes and quantization config are identical. In a spot check of one full layer weight, about 0.2% of the 4-bit codes differ by one or two steps from floating-point rounding, and both copies have the same error versus the bf16 original.
Author
Quantized and benchmarked by Pratik Gandhi (@pratikg on X).
Write-up on GitHub
How I ran this, what broke, and the scripts: Nanbeige4.3-33B on my Mac mini. It's part of Local AI field notes, where I log every model I run on my Mac mini.