4-bit weights and activations: 22.07 GB, 336 tok/s at concurrency 32 on one 96 GB card.
NVFP4 build of Nanbeige/Nanbeige4.3-33B at 22.07 GB, against 66.24 GB for the BF16 release.
Every linear layer in the 52 decoder blocks is NVFP4: E2M1 weights with an E4M3 scale per 16 values and an FP32 global scale, activations quantized to NVFP4 at runtime against a calibrated global scale. Embeddings, the LM head and the norms stay BF16.
Drift is the extra negative log-likelihood per token a build assigns to 400 reasoning traces that BF16 wrote on the same card; top-1 agreement is how often its most likely next token is BF16's. The pass is deterministic.
Why this build
- ⚡ 53.8 tok/s single-stream and 336 at concurrency 32 on one card, against 20.1 and 129 for BF16 on the same card.
- 🗜️ 22.07 GB, against 66.24 GB in BF16. At a 32,768-token window the KV pool is 326,144 tokens, against 113,648 for BF16.
- 📏 Knowledge 89.0, tool calls 78.8, abstain 64.2; BF16 scores 89.7, 80.8 and 63.3. 99.1% of answers finish inside the budget.
- 📉 What 4 bits cost: drift +64.3×10⁻³ nats per token, top-1 agreement 93.7%, against +5.95 and 97.9% for FP8. The mixed build brings that to +38.9 for 3.23 GB more.
Serve it
hf download primitive-ai/Nanbeige4.3-33B-NVFP4 --local-dir ./Nanbeige4.3-33B-NVFP4
docker run --gpus '"device=0"' --ipc=host -p 8000:8000 -v $PWD:/models \
vllm/vllm-openai:v0.32.0 \
--model /models/Nanbeige4.3-33B-NVFP4 \
--max-model-len 131072 --gpu-memory-utilization 0.92 \
--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder
Checked with exactly this command on an idle card: boot 191 s, 20.92 GiB of weights, a 313,584-token KV pool at the 131,072-token window. What is 17*23? comes back as 391 with the reasoning split off by the parser, and as 391 with chat_template_kwargs: {"enable_thinking": false}; a weather question comes back as the parsed call get_weather {"city": "Lisbon"}. The model's window is 524,288 tokens; raise --max-model-len as far as the KV pool allows.
Measured
One RTX PRO 6000 Blackwell, 96 GB, no tensor parallelism, vLLM 0.32.0, BF16 KV cache. The 1,170-item knowledge suite and the 200-item tool-calling suite, temperature 0.6 / top_p 0.95 / top_k 20, thinking on, a 16,384-token budget, concurrency 32, auto-scored with no LLM judge. Throughput is 8K in / 512 out, prefix-cache free, two seeds per cell. KV pool is at a 32,768-token window and --gpu-memory-utilization 0.92.
| build | format | size | drift (×10⁻³ nats/token) | top-1 agreement | knowledge | call | abstain | finished | tok/s @1 | tok/s @32 | KV pool (tokens) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Nanbeige4.3-33B (BF16 release) | BF16 | 66.24 GB | 0 (reference) | 100% | 89.7 | 80.8 | 63.3 | 99.2% | 20.1 | 129 | 113,648 |
| FP8 | FP8 W8A8, every Linear | 35.53 GB | +5.95 | 97.9% | 89.3 | 80.4 | 63.3 | 99.1% | 35.4 | 225 | 262,448 |
| Mixed NVFP4 / FP8 | NVFP4 W4A4, FP8 W8A8 in 216 modules | 25.30 GB | +38.89 | 95.1% | 88.9 | 79.0 | 63.3 | 99.1% | 48.0 | 308 | 311,488 |
| this repo | NVFP4 W4A4, every Linear | 22.07 GB | +64.28 | 93.7% | 89.0 | 78.8 | 64.2 | 99.1% | 53.8 | 336 | 326,144 |
Drift and agreement come from 400 BF16 traces of the knowledge suite (215,950 completion tokens); the standard error over traces is 1.6 for this build. Tool calling is 3 runs per build; knowledge is 2 runs per quantized build and 1 for BF16.
What this table does not cover:
- Context past 32K. The model's window is 524,288 tokens; we did not measure the quantized layers there.
- Other GPUs. Only the RTX PRO 6000 Blackwell (sm_120) was tested.
Comparable with our other models
Accuracy numbers move for reasons that have nothing to do with the model: a shorter token budget, a
different temperature, or whether the model was allowed to reason at all. So every number in this
table, on this card and on our other cards, comes from the one fixed protocol described above, the
same 1,370 items, auto-scored, no LLM judge.
| model | shape | size | overall | knowledge | call | abstain | finished | out/answer |
|---|---|---|---|---|---|---|---|---|
| Laguna-XS-2.1 | 31 B MoE | 19.3 GiB | 81.7 | 83.8 | 68.4 | 73.5 | 98.9% | 1097 |
| Nemotron-3.5-Lightning-30B-A3B | 30 B MoE+Mamba | 19.2 GiB | 87.1 | 87.9 | 85.4 | 70.5 | 97.9% | 1429 |
| Ornith-1.5-35B-A3B | 35 B MoE | 22.6 GiB | 88.7 | 91.7 | 74.4 | 60.0 | 99.3% | 760 |
| Muse-Glimmer-30B | 30 B MoE | 20.4 GiB | 86.6 | 88.8 | 78.6 | 54.5 | 99.7% | 800 |
| Qwen3.8-27B | 27 B dense | 20.7 GiB | 88.8 | 90.4 | 85.5 | 54.5 | 99.7% | 651 |
| Granite-4.2-30B | 30 B dense | 18.1 GB | 85.5 | 86.2 | 85.8 | 60.8 | 98.5% | 1502 |
| Nex-N2.5-mini NVFP4 | 35 B MoE, 3 B active | 23.91 GB | 88.5 | 90.5 | 81.9 | 57.5 | 99.3% | 524 |
| Nex-N2.5-mini mixed | 35 B MoE, 3 B active | 26.04 GB | 88.5 | 90.6 | 80.9 | 58.7 | 99.2% | 504 |
| Nex-N2.5-mini FP8 | 35 B MoE, 3 B active | 38.13 GB | 88.7 | 90.9 | 79.4 | 60.0 | 99.5% | 545 |
| K2-Horizon-MoVA-36B-A4B NVFP4 | 37 B MoE+MoVA, 4 B active | 36.7 GB | 84.1 | 86.5 | 71.8 | 60.4 | 95.6% | 1234 |
| K2-Horizon-MoVA-36B-A4B mixed | 37 B MoE+MoVA, 4 B active | 44.5 GB | 84.9 | 87.3 | 73.5 | 58.9 | 96.3% | 1118 |
| Nemotron-3.5-Bolt-74B-A9B mixed | 75 B MoE+Mamba, 9 B active | 46.70 GB | 89.3 | 90.6 | 87.9 | 58.3 | 98.5% | 1173 |
| Laguna-S-2.1 | 110 B MoE | 64.0 GiB | 84.3 | 87.1 | 64.6 | 81.0 | 97.3% | 995 |
| Qwen3.8-Flash-Next | 180 B MoE, 6 B active | 183.7 GB | 90.3 | 92.2 | 84.8 | 56.7 | 99.5% | 686 |
| MiMo-V2.6-Flash-RL NVFP4 | 309 B MoE, 15 B active | 182.40 GB | 89.3 | 91.1 | 83.0 | 61.0 | 99.0% | 664 |
| Kolibri-1 FP8 | 78 B MoE, 3.5 B active | 78.83 GB | 85.0 | 86.1 | 86.1 | 48.0 | 97.0% | 1824 |
| Kolibri-1 NVFP4 | 78 B MoE, 3.5 B active | 47.70 GB | 84.9 | 86.2 | 84.9 | 48.0 | 96.4% | 1947 |
| Kolibri-1 mixed | 78 B MoE, 3.5 B active | 45.81 GB | 83.9 | 85.0 | 85.5 | 47.5 | 96.6% | 1848 |
| LFM2.5-24B-A2B NVFP4A16 | 24 B MoE, 2.45 B active | 14.95 GB | 83.9 | 85.4 | 82.4 | 44.5 | 98.8% | 1603 |
| MiniMax-M3.1-Flash (release, 4 cards) | 427 B MoE, 23 B active | 250.12 GB | 82.9 | 85.0 | 81.7 | 25.0 | 99.2% | 513 |
| MiniMax-M3.1-Flash-pruned-272B | 272 B MoE, 23 B active | 162.90 GB | 82.5 | 84.4 | 80.4 | 32.5 | 98.8% | 707 |
| Nanbeige4.3-33B (BF16 release) | 33 B dense | 66.24 GB | 87.9 | 89.7 | 80.8 | 63.3 | 99.2% | 846 |
| Nanbeige4.3-33B FP8 | 33 B dense | 35.53 GB | 87.5 | 89.3 | 80.4 | 63.3 | 99.1% | 871 |
| Nanbeige4.3-33B mixed | 33 B dense | 25.30 GB | 87.0 | 88.9 | 79.0 | 63.3 | 99.1% | 905 |
| Nanbeige4.3-33B NVFP4 (this repo) | 33 B dense | 22.07 GB | 87.1 | 89.0 | 78.8 | 64.2 | 99.1% | 940 |
overall pools the two suites as 1,370 items, weighted 85.4% knowledge and 14.6% tool calling by
item count. Read it with finished: overall scores an answer that overran the token budget as
wrong, and cannot say whether the model needed the room or failed to stop. A gap under 1.0 in
overall is a tie. Sizes are as each card reports them, which mixes GB and GiB.
What's in it
Nanbeige4.3-33B is a dense Llama-architecture model: 33.12 B parameters, 52 layers, hidden size 7168, 56 query and 8 KV heads, MLP width 22,016.
| params | share | |
|---|---|---|
| MLP gate + up + down, 52 layers | 24.62 B | 74.3% |
| attention q + k + v, 52 layers | 3.44 B | 10.4% |
| attention o, 52 layers | 2.67 B | 8.1% |
| embeddings and LM head (untied) | 2.39 B | 7.2% |
| modules | count | format |
|---|---|---|
| every decoder Linear (q, k, v, o, gate, up, down) | 364 | NVFP4 (E2M1, E4M3 scale per 16, FP32 global); activations NVFP4 per 16 |
| embeddings, LM head, norms | all | BF16, byte-identical to the release |
The checkpoint is compressed-tensors and runs on vLLM's stock kernels. q/k/v and gate/up share one format per layer because vLLM fuses them.
Licence
Apache-2.0, the base model's licence. This is a modified version of Nanbeige/Nanbeige4.3-33B: the decoder's linear layers were quantized and config.json carries a quantization_config. Everything else is unchanged.
primitive ·
more models ·
inference economics for production LLM systems
LocalMaxxing