primitive-ai/Kolibri-1-NVFP4

Level with Aleph Alpha's FP8 release on accuracy, at 60% of its size.

NVFP4 build of Aleph-Alpha/Kolibri-1-BF16 at 47.70 GB, on one 96 GB card.
All 50 layers of routed experts at 4 bits; attention, shared experts, embeddings, lm_head, routers and norms stay BF16.

Two builds, one measurement. We quantized the experts two ways and ran both against Aleph Alpha's own FP8 release under one protocol. This build is the more accurate on knowledge. The mixed build keeps the release's FP8 attention instead of BF16, and is the faster of the two. On tool calling all three are one band.

---

Why this quant

• 🗜️ 47.70 GB against 78.83 GB for the FP8 release. Weights take 44.53 GiB of the card against 73.55 GiB, and the KV pool at 32K context goes from 410,315 tokens to 1,400,826 (3.4×).
• 🎯 Level with the release. Knowledge 86.2 against 86.1, tool calling 77.5 against 78.5 over five runs each. Both gaps sit inside the run-to-run spread.
• ⚡ 1,076 tok/s at concurrency 32, against 986 for the release (+9%). Single-stream it is slower: 124.4 tok/s against 159.1.
• 🐢 BF16 attention costs single-stream speed. The mixed build, which keeps the release's FP8 attention and shared experts, decodes 15% faster at concurrency 1 and 11% faster at 32, and is 1.89 GB smaller. It gives up about a point of knowledge for that.
• 🧩 Stock Aleph Alpha image, nothing patched. compressed-tensors, nvfp4-pack-quantized.

---

Serve it

hf download primitive-ai/Kolibri-1-NVFP4 --local-dir ./Kolibri-1-NVFP4

docker run --gpus all --ipc=host -p 8000:8000 -v $PWD:/models \
  ghcr.io/aleph-alpha/aleph-alpha-inference:1.0.0-vllm0.29.0 \
  /models/Kolibri-1-NVFP4 --served-model-name kolibri \
  --kv-cache-dtype fp8 --max-model-len 32768 --gpu-memory-utilization 0.92 \
  --reasoning-parser kolibri1 --tool-call-parser kolibri1 --enable-auto-tool-choice

Stock vLLM does not know the Kolibri1ForCausalLM architecture. Aleph Alpha's plugin image adds it along with the kolibri1 parsers, and nothing here is patched on top of it. The image is vLLM 0.29.0 with aleph-alpha-inference 1.0.0 (sha256:9a56ab1691f8bc8eb0fc82f0f534bfdd36da2218fa79b01b83448739fbda403b). Boot takes about 151 s on an idle card, and the NVFP4 experts run on FlashInfer CUTLASS.

Aleph Alpha recommends temperature 1.0, top_p 0.97, top_k 128. Our numbers use the fixed protocol above for every build, so they are comparable with each other, not with Aleph Alpha's own. The measurement servers ran without the two parsers, because the tool-calling suite does not pass tools=. Thinking is on by default; send enable_thinking: false to turn it off.

---

Measured

One RTX PRO 6000 Blackwell, 96 GB, one card. The 1,170-item knowledge suite and the 200-item tool-calling suite, temperature 0.6 / top_p 0.95 / top_k 20, thinking on, a 16,384-token budget, concurrency 32, auto-scored with no LLM judge. FP8 KV cache for every row, which is how Aleph Alpha evaluates the release. Throughput is 8K in / 512 out, prefix-cache free, two seeds per cell. Every row ran on the same box on the same day.

Tool-calling is the mean of five runs per build, with a within-build standard deviation of 0.6 to 1.2, so that column is one band. Knowledge is two runs per build (86.0/86.2, 84.8/85.1, 85.9/86.5 strict, in table order). On knowledge the release and NVFP4 are 0.1 apart; the mixed build is 1.1 below the release and 1.2 below NVFP4, on both of its runs. KV pool is the number vLLM reports at --max-model-len 32768, --gpu-memory-utilization 0.92.

What this table does not cover:

• The BF16 original. It is 156.2 GB and does not fit on one card, so Aleph Alpha's FP8 release is the reference row.
• German. Both suites are English. Kolibri is built for German and English, and we did not measure German.
• Context past 32K, and anything but Blackwell. The release is validated to 1M tokens; we did not test the 4-bit experts there. NVFP4 runs natively on Blackwell only.

---

Comparable with our other models

Accuracy numbers move for reasons that have nothing to do with the model: a shorter token budget, a different temperature, or whether the model was allowed to reason at all. So every number in this table, on this card and on our other cards, comes from the one fixed protocol described above, the same 1,370 items, auto-scored, no LLM judge.

overall pools the two suites as 1,370 items, weighted 85.4% knowledge and 14.6% tool calling by item count. Read it with finished: overall scores an answer that overran the token budget as wrong, and cannot say whether the model needed the room or failed to stop. A gap under 1.0 in overall is a tie. Sizes are as each card reports them, which mixes GB and GiB.

---

What's quantized to what

Kolibri-1 is 78.10 B parameters, 3.46 B active per token, and 96.7% of them are routed experts.

compressed-tensors, format nvfp4-pack-quantized, one config group, W4A4 with group-16 weight scales and a per-module global scale. The source already ships experts as per-expert modules, so there is nothing to unfold. gate and up of every expert share one weight_global_scale; vLLM fuses those two halves and keeps one scale, warning and taking the maximum when they disagree, and loading logged no such warning on any layer. All 58,353 tensors of the BF16 source are accounted for. Expert weights are round-to-nearest, with no calibration.

---

_{primitive · more models · inference economics for production LLM systems}