Nanbeige4.3-33B · MLX 8-bit
Nanbeige/Nanbeige4.3-33B (revision c1093c50) for Apple silicon:
every linear layer, the embedding and the LM head in MLX affine 8-bit, group 64. 35.2 GB, 8.50 bits per weight,
98.6% of next tokens the same as bf16's. Runs in mlx-lm and in
Kapitsa.
| Base model | Nanbeige4.3-33B: dense Llama, 52 layers, 56 query / 8 KV heads, 33.1B parameters, 512K context |
| Format | MLX safetensors, affine 8-bit, group size 64, bf16 scales and biases |
| Size | 35.2 GB on disk, 8.50 bits per weight |
| Memory | a 48 GB Mac with a short context; 64 GB for 64K tokens; 96 GB for 128K (the KV cache is 208 KB a token in fp16) |
| Other packs | 4-bit AWQ (19.8 GB) · bf16 (66.3 GB) |
Quality
Every pack below scored on the same test against the bf16 model: 768 next-token positions in 8 windows of Python
source (512-3,224 tokens, each starting with <|im_start|> as a chat does), computed by Kapitsa on an M5 Max. The
GGUFs were made with llama.cpp b11871 from the same checkpoint and dequantized back to bf16 so they ran through the
same engine.
| Pack | Size | Bits/weight | KL mean | KL median | KL p99 | Same top token |
|---|---|---|---|---|---|---|
| this pack: MLX 8-bit | 35.2 GB | 8.50 | 0.0155 | 0.00003 | 0.18 | 98.6% |
| llama.cpp Q8_0 | 35.2 GB | 8.50 | 0.0059 | 0.00001 | 0.12 | 99.0% |
| MLX 4-bit AWQ (ours) | 19.8 GB | 4.79 | 0.163 | 0.0015 | 3.77 | 92.3% |
| llama.cpp Q4_K_M | 20.1 GB | 4.84 | 0.144 | 0.0010 | 2.65 | 93.5% |
At the same size, Q8_0 sits closer to bf16. Its blocks of 32 weights share one fp16 scale; MLX's groups of 64 store a bf16 scale and bias, and the bias's rounding is about half an 8-bit step. A bf16 LM head on top of this pack measured the same (KL 0.0157), so the gap is in the layers. KL is measured in nats per position.
Speed
MacBook Pro M5 Max (128 GB, macOS 26.4). Prefill of a prompt, then decode at that context (tokens/s). Kapitsa and mlx-lm 0.32.2 ran this pack; llama.cpp b11871 ran a Q8_0 of the same checkpoint. Each engine ran in turn at each length, with the GPU quiet before each run.
| Context | prefill: Kapitsa / mlx-lm / llama.cpp | decode: Kapitsa / mlx-lm / llama.cpp |
|---|---|---|
| 1K | 665 / 725 / 708 | 15.6 / 15.8 / 16.3 |
| 4K | 652 / 699 / 575 | 15.1 / 15.3 / 15.8 |
| 16K | 510 / 461 / 360 | 13.9 / 13.7 / 12.8 |
| 32K | 397 / 433 / 311 | 12.8 / 12.7 / 12.4 |
Runs swing by up to about 20% as the chip heats.
The runs, logs and scripts are in Kapitsa's repository.
Run
Kapitsa 0.2.18 or later (Apple silicon, Metal):
brew install nace-ai/tap/kapitsa # or: curl -fsSL https://raw.githubusercontent.com/nace-ai/kapitsa/main/install.sh | bash
kapitsa serve --model nanbeige:8bit # downloads this pack, serves an OpenAI-compatible API on localhost
mlx-lm:
pip install mlx-lm
mlx_lm.generate --model kapitsaprime/Nanbeige4.3-33B-MLX-8bit --prompt "Write a Python function that merges two sorted lists."
mlx_lm.server --model kapitsaprime/Nanbeige4.3-33B-MLX-8bit
The chat template, tokenizer and generation settings are the original model's. Reasoning goes in <think> and is on
by default; enable_thinking: false in the template arguments turns it off. Tool calls are XML in <tool_call>.
Nanbeige recommends temperature 0.6 and top-p 0.95 for reasoning and chat, 1.0 and 0.95 for agentic use.
How it was made
mlx_lm.convert --hf-path Nanbeige/Nanbeige4.3-33B --mlx-path out -q --q-bits 8 --q-group-size 64
(mlx-lm 0.32.2, mlx 0.32.4.)
License and credits
Apache 2.0, as the base model. Weights by Nanbeige; quantized and measured by Nace AI with Kapitsa.