4-bit weights and 16-bit activations in 37.9 GB, attention layers included.
NVFP4A16 build of ricursiveAI/Ripple-65B-Preview at 37.92 GB, against 129.57 GB for the BF16 release.
Every decoder Linear stores NVFP4 weights (E2M1 with an E4M3 scale per 16 values and an FP32 global scale); activations stay BF16. Embeddings, the LM head, the norms, the per-head Q/K norms and the value-residual scalars stay BF16.
No vLLM release runs this architecture yet. Support is an open pull request, vllm-project/vllm#65745. Every number on this card was measured with vLLM
nightly-a2bcb5657a88plus that PR'svllm/files (headd53dfbc2c76d).Drift is the extra negative log-likelihood per token a build assigns to 400 reasoning traces that BF16 wrote; top-1 agreement is how often its most likely next token is BF16's. The pass is deterministic.
Why this build
- 🗜️ 37.92 GB, against 57.22 GB for Ricursive's NVFP4 and 129.57 GB in BF16. Attention is 4-bit here as well; in Ricursive's NVFP4 it stays BF16.
- 🎯 Drift +14.63×10⁻³ nats per token from BF16, top-1 agreement 96.5%, against +5.49 and 97.3% for Ricursive's NVFP4 and +1.72 for FP8.
- 📏 Knowledge 90.3, tool calls 84.6, abstain 56.7; BF16 scores 90.5, 86.2 and 56.7. 99.4% of answers finish inside the budget.
- ⚡ 26.3 tok/s single-stream and 110 at concurrency 32 on one card, against 21.0 and 180 for Ricursive's NVFP4 on the same card. BF16 does not fit one card (129.6 GB), so it has no one-card row.
Serve it
hf download primitive-ai/Ripple-65B-Preview-NVFP4A16 --local-dir ./Ripple-65B-Preview-NVFP4A16
docker run --gpus '"device=0"' --ipc=host -p 8000:8000 -v $PWD:/models \
vllm-ripple:pr65745 \
--model /models/Ripple-65B-Preview-NVFP4A16 \
--max-model-len 131072 --gpu-memory-utilization 0.92 \
--reasoning-parser ripple --enable-auto-tool-choice --tool-call-parser ripple
Checked with exactly these flags on an idle card. It boots in 191 s, loads 36.09 GiB of weights and gets a 497,045-token KV pool at the 131,072-token window. What is 17*23? comes back as 391 with the reasoning split off by the parser, and a weather question comes back as the parsed call get_weather {"city": "Lisbon"}. The model's window is 2,097,152 positions; raise --max-model-len as far as the KV pool allows.
vllm-ripple:pr65745 is not a published image. Build it with vllm_ripple/build.sh in this repository, which applies the pull request's files to that nightly. Delete that step once #65745 is in a release.
Measured
One RTX PRO 6000 Blackwell, 96 GB, no tensor parallelism, BF16 KV cache. The 1,170-item knowledge suite and the 200-item tool-calling suite, temperature 0.6 / top_p 0.95 / top_k 20, a 16,384-token budget, concurrency 32, auto-scored with no LLM judge; the model opens its own <think> block and the chat template has no switch. Throughput is 8K in / 512 out, prefix-cache free, two seeds per cell. KV pool is at a 32,768-token window and --gpu-memory-utilization 0.92.
| build | format | size | drift (×10⁻³ nats/token) | top-1 agreement | knowledge | call | abstain | finished | tok/s @1 | tok/s @32 | KV pool (tokens) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Ripple-65B-Preview (BF16 release) | BF16 | 129.57 GB | 0 (reference) | 100% | 90.5 | 86.2 | 56.7 | 99.4% | n/a | n/a | n/a |
| Ricursive NVFP4 | NVFP4 W4A4 MLP, attention BF16 | 57.22 GB | +5.49 | 97.3% | 90.4 | 85.4 | 58.3 | 99.4% | 21.0 | 180 | 157,351 |
| FP8 | FP8 W8A8, every decoder Linear | 65.85 GB | +1.72 | 98.6% | 91.0 | 85.4 | 61.7 | 99.3% | 18.0 | 138 | 118,546 |
| this repo | NVFP4 W4A16, every decoder Linear | 37.92 GB | +14.63 | 96.5% | 90.3 | 84.6 | 56.7 | 99.4% | 26.3 | 110 | 251,501 |
BF16 runs on two cards (tensor parallel 2) because it does not fit one, and it is the reference every other build's drift is measured against. Drift and agreement come from 400 traces (185,735 completion tokens); the standard error over traces is 0.66 for this build. Tool calling is 3 runs per build, knowledge 1.
What this table does not cover:
- Context past 32K. The model's window is 2,097,152 positions; every number here is at 32,768 or less.
- Other GPUs. Only the RTX PRO 6000 Blackwell (sm_120) was tested.
Comparable with our other models
Accuracy numbers move for reasons that have nothing to do with the model: a shorter token budget, a
different temperature, or whether the model was allowed to reason at all. So every number in this
table, on this card and on our other cards, comes from the one fixed protocol described above, the
same 1,370 items, auto-scored, no LLM judge.
| model | shape | size | overall | knowledge | call | abstain | finished | out/answer |
|---|---|---|---|---|---|---|---|---|
| Laguna-XS-2.1 | 31 B MoE | 19.3 GiB | 81.7 | 83.8 | 68.4 | 73.5 | 98.9% | 1097 |
| Nemotron-3.5-Lightning-30B-A3B | 30 B MoE+Mamba | 19.2 GiB | 87.1 | 87.9 | 85.4 | 70.5 | 97.9% | 1429 |
| Ornith-1.5-35B-A3B | 35 B MoE | 22.6 GiB | 88.7 | 91.7 | 74.4 | 60.0 | 99.3% | 760 |
| Muse-Glimmer-30B | 30 B MoE | 20.4 GiB | 86.6 | 88.8 | 78.6 | 54.5 | 99.7% | 800 |
| Qwen3.8-27B | 27 B dense | 20.7 GiB | 88.8 | 90.4 | 85.5 | 54.5 | 99.7% | 651 |
| Granite-4.2-30B | 30 B dense | 18.1 GB | 85.5 | 86.2 | 85.8 | 60.8 | 98.5% | 1502 |
| Nex-N2.5-mini NVFP4 | 35 B MoE, 3 B active | 23.91 GB | 88.5 | 90.5 | 81.9 | 57.5 | 99.3% | 524 |
| Nex-N2.5-mini mixed | 35 B MoE, 3 B active | 26.04 GB | 88.5 | 90.6 | 80.9 | 58.7 | 99.2% | 504 |
| Nex-N2.5-mini FP8 | 35 B MoE, 3 B active | 38.13 GB | 88.7 | 90.9 | 79.4 | 60.0 | 99.5% | 545 |
| K2-Horizon-MoVA-36B-A4B NVFP4 | 37 B MoE+MoVA, 4 B active | 36.7 GB | 84.1 | 86.5 | 71.8 | 60.4 | 95.6% | 1234 |
| K2-Horizon-MoVA-36B-A4B mixed | 37 B MoE+MoVA, 4 B active | 44.5 GB | 84.9 | 87.3 | 73.5 | 58.9 | 96.3% | 1118 |
| Nemotron-3.5-Bolt-74B-A9B mixed | 75 B MoE+Mamba, 9 B active | 46.70 GB | 89.3 | 90.6 | 87.9 | 58.3 | 98.5% | 1173 |
| Laguna-S-2.1 | 110 B MoE | 64.0 GiB | 84.3 | 87.1 | 64.6 | 81.0 | 97.3% | 995 |
| Qwen3.8-Flash-Next | 180 B MoE, 6 B active | 183.7 GB | 90.3 | 92.2 | 84.8 | 56.7 | 99.5% | 686 |
| MiMo-V2.6-Flash-RL NVFP4 | 309 B MoE, 15 B active | 182.40 GB | 89.3 | 91.1 | 83.0 | 61.0 | 99.0% | 664 |
| Kolibri-1 FP8 | 78 B MoE, 3.5 B active | 78.83 GB | 85.0 | 86.1 | 86.1 | 48.0 | 97.0% | 1824 |
| Kolibri-1 NVFP4 | 78 B MoE, 3.5 B active | 47.70 GB | 84.9 | 86.2 | 84.9 | 48.0 | 96.4% | 1947 |
| Kolibri-1 mixed | 78 B MoE, 3.5 B active | 45.81 GB | 83.9 | 85.0 | 85.5 | 47.5 | 96.6% | 1848 |
| LFM2.5-24B-A2B NVFP4A16 | 24 B MoE, 2.45 B active | 14.95 GB | 83.9 | 85.4 | 82.4 | 44.5 | 98.8% | 1603 |
| MiniMax-M3.1-Flash (release, 4 cards) | 427 B MoE, 23 B active | 250.12 GB | 82.9 | 85.0 | 81.7 | 25.0 | 99.2% | 513 |
| MiniMax-M3.1-Flash-pruned-285B (this repo) | 285 B MoE, 23 B active | 170.17 GB | 80.6 | 82.3 | 80.8 | 28.3 | 97.7% | 932 |
| Ripple-65B-Preview (BF16, 2 cards) | 65 B dense | 129.57 GB | 89.0 | 90.5 | 86.2 | 56.7 | 99.4% | 751 |
| Ripple-65B-Preview NVFP4 (Ricursive) | 65 B dense | 57.22 GB | 88.9 | 90.4 | 85.4 | 58.3 | 99.4% | 758 |
| Ripple-65B-Preview FP8 | 65 B dense | 65.85 GB | 89.5 | 91.0 | 85.4 | 61.7 | 99.3% | 756 |
| Ripple-65B-Preview NVFP4A16 (this repo) | 65 B dense | 37.92 GB | 88.6 | 90.3 | 84.6 | 56.7 | 99.4% | 832 |
overall pools the two suites as 1,370 items, weighted 85.4% knowledge and 14.6% tool calling by
item count. Read it with finished: overall scores an answer that overran the token budget as
wrong, and cannot say whether the model needed the room or failed to stop. A gap under 1.0 in
overall is a tie. Sizes are as each card reports them, which mixes GB and GiB.
What's in it
Ripple-65B-Preview is a dense 65 B model (80 layers, hidden size 5120, 80 query and 8 KV heads, MLP width 40,960).
| params | share | |
|---|---|---|
| MLP gate + up + down, 80 layers | 50.33 B | 77.6% |
attention q + k + g + v + o, 80 layers (g is the output gate) |
13.42 B | 20.7% |
| embeddings and LM head (untied) | 1.03 B | 1.6% |
| modules | count | format |
|---|---|---|
| every decoder Linear (q, k, g, v, o, gate, up, down) | 640 | NVFP4 weights (E2M1, E4M3 scale per 16, FP32 global); activations BF16 |
| embeddings, LM head, norms, Q/K norms, value-residual scalars | all | BF16, byte-identical to the release |
The checkpoint is compressed-tensors. vLLM fuses q/k/g into one module and gate/up into another, so each fused group shares one format and one global scale.
Licence
Apache-2.0, the base model's licence. This is a modified version of ricursiveAI/Ripple-65B-Preview: the decoder's linear layers were quantized and config.json carries a quantization_config. Everything else is unchanged.
primitive ·
more models ·
inference economics for production LLM systems