primitive-ai/Ripple-65B-Preview-FP8

8-bit weights and activations: 65.8 GB, +1.7×10⁻³ drift from BF16, on one 96 GB card.

FP8 build of ricursiveAI/Ripple-65B-Preview at 65.85 GB, against 129.57 GB for the BF16 release.
Every decoder Linear stores FP8 E4M3 weights with one scale per output channel; activations are quantized to FP8 per token at runtime. Embeddings, the LM head, the norms, the per-head Q/K norms and the value-residual scalars stay BF16.

No vLLM release runs this architecture yet. Support is an open pull request, vllm-project/vllm#65745. Every number on this card was measured with vLLM nightly-a2bcb5657a88 plus that PR's vllm/ files (head d53dfbc2c76d).

Drift is the extra negative log-likelihood per token a build assigns to 400 reasoning traces that BF16 wrote; top-1 agreement is how often its most likely next token is BF16's. The pass is deterministic.

Why this build

🎯 Drift +1.72×10⁻³ nats per token from BF16, top-1 agreement 98.6%. Ricursive's NVFP4 is +5.49 and 97.3%.
📏 Knowledge 91.0, tool calls 85.4, abstain 61.7; BF16 scores 90.5, 86.2 and 56.7. 99.3% of answers finish inside the budget.
⚡ 18.0 tok/s single-stream and 138 at concurrency 32 on one card, against 21.0 and 180 for Ricursive's NVFP4 on the same card. BF16 does not fit one card (129.6 GB), so it has no one-card row.
🗜️ 65.85 GB, against 129.57 GB in BF16: one 96 GB card holds it with a 118,546-token KV pool at a 32,768-token window.

Serve it

hf download primitive-ai/Ripple-65B-Preview-FP8 --local-dir ./Ripple-65B-Preview-FP8

docker run --gpus '"device=0"' --ipc=host -p 8000:8000 -v $PWD:/models \
  vllm-ripple:pr65745 \
  --model /models/Ripple-65B-Preview-FP8 \
  --max-model-len 131072 --gpu-memory-utilization 0.92 \
  --reasoning-parser ripple --enable-auto-tool-choice --tool-call-parser ripple

Checked with exactly these flags on an idle card. It boots in 201 s, loads 61.90 GiB of weights and gets a 225,291-token KV pool at the 131,072-token window. What is 17*23? comes back as 391 with the reasoning split off by the parser, and a weather question comes back as the parsed call get_weather {"city": "Lisbon"}. The model's window is 2,097,152 positions; raise --max-model-len as far as the KV pool allows.

vllm-ripple:pr65745 is not a published image. Build it with vllm_ripple/build.sh in this repository, which applies the pull request's files to that nightly. Delete that step once #65745 is in a release.

Measured

One RTX PRO 6000 Blackwell, 96 GB, no tensor parallelism, BF16 KV cache. The 1,170-item knowledge suite and the 200-item tool-calling suite, temperature 0.6 / top_p 0.95 / top_k 20, a 16,384-token budget, concurrency 32, auto-scored with no LLM judge; the model opens its own <think> block and the chat template has no switch. Throughput is 8K in / 512 out, prefix-cache free, two seeds per cell. KV pool is at a 32,768-token window and --gpu-memory-utilization 0.92.

build	format	size	drift (×10⁻³ nats/token)	top-1 agreement	knowledge	call	abstain	finished	tok/s @1	tok/s @32	KV pool (tokens)
Ripple-65B-Preview (BF16 release)	BF16	129.57 GB	0 (reference)	100%	90.5	86.2	56.7	99.4%	n/a	n/a	n/a
Ricursive NVFP4	NVFP4 W4A4 MLP, attention BF16	57.22 GB	+5.49	97.3%	90.4	85.4	58.3	99.4%	21.0	180	157,351
this repo	FP8 W8A8, every decoder Linear	65.85 GB	+1.72	98.6%	91.0	85.4	61.7	99.3%	18.0	138	118,546
NVFP4A16	NVFP4 W4A16, every decoder Linear	37.92 GB	+14.63	96.5%	90.3	84.6	56.7	99.4%	26.3	110	251,501

BF16 runs on two cards (tensor parallel 2) because it does not fit one, and it is the reference every other build's drift is measured against. Drift and agreement come from 400 traces (185,735 completion tokens); the standard error over traces is 0.16 for this build. Tool calling is 3 runs per build, knowledge 1.

What this table does not cover:

Context past 32K. The model's window is 2,097,152 positions; every number here is at 32,768 or less.
Other GPUs. Only the RTX PRO 6000 Blackwell (sm_120) was tested.

Comparable with our other models

Accuracy numbers move for reasons that have nothing to do with the model: a shorter token budget, a different temperature, or whether the model was allowed to reason at all. So every number in this table, on this card and on our other cards, comes from the one fixed protocol described above, the same 1,370 items, auto-scored, no LLM judge.

model	shape	size	overall	knowledge	call	abstain	finished	out/answer
Laguna-XS-2.1	31 B MoE	19.3 GiB	81.7	83.8	68.4	73.5	98.9%	1097
Nemotron-3.5-Lightning-30B-A3B	30 B MoE+Mamba	19.2 GiB	87.1	87.9	85.4	70.5	97.9%	1429
Ornith-1.5-35B-A3B	35 B MoE	22.6 GiB	88.7	91.7	74.4	60.0	99.3%	760
Muse-Glimmer-30B	30 B MoE	20.4 GiB	86.6	88.8	78.6	54.5	99.7%	800
Qwen3.8-27B	27 B dense	20.7 GiB	88.8	90.4	85.5	54.5	99.7%	651
Granite-4.2-30B	30 B dense	18.1 GB	85.5	86.2	85.8	60.8	98.5%	1502
Nex-N2.5-mini NVFP4	35 B MoE, 3 B active	23.91 GB	88.5	90.5	81.9	57.5	99.3%	524
Nex-N2.5-mini mixed	35 B MoE, 3 B active	26.04 GB	88.5	90.6	80.9	58.7	99.2%	504
Nex-N2.5-mini FP8	35 B MoE, 3 B active	38.13 GB	88.7	90.9	79.4	60.0	99.5%	545
K2-Horizon-MoVA-36B-A4B NVFP4	37 B MoE+MoVA, 4 B active	36.7 GB	84.1	86.5	71.8	60.4	95.6%	1234
K2-Horizon-MoVA-36B-A4B mixed	37 B MoE+MoVA, 4 B active	44.5 GB	84.9	87.3	73.5	58.9	96.3%	1118
Nemotron-3.5-Bolt-74B-A9B mixed	75 B MoE+Mamba, 9 B active	46.70 GB	89.3	90.6	87.9	58.3	98.5%	1173
Laguna-S-2.1	110 B MoE	64.0 GiB	84.3	87.1	64.6	81.0	97.3%	995
Qwen3.8-Flash-Next	180 B MoE, 6 B active	183.7 GB	90.3	92.2	84.8	56.7	99.5%	686
MiMo-V2.6-Flash-RL NVFP4	309 B MoE, 15 B active	182.40 GB	89.3	91.1	83.0	61.0	99.0%	664
Kolibri-1 FP8	78 B MoE, 3.5 B active	78.83 GB	85.0	86.1	86.1	48.0	97.0%	1824
Kolibri-1 NVFP4	78 B MoE, 3.5 B active	47.70 GB	84.9	86.2	84.9	48.0	96.4%	1947
Kolibri-1 mixed	78 B MoE, 3.5 B active	45.81 GB	83.9	85.0	85.5	47.5	96.6%	1848
LFM2.5-24B-A2B NVFP4A16	24 B MoE, 2.45 B active	14.95 GB	83.9	85.4	82.4	44.5	98.8%	1603
MiniMax-M3.1-Flash (release, 4 cards)	427 B MoE, 23 B active	250.12 GB	82.9	85.0	81.7	25.0	99.2%	513
MiniMax-M3.1-Flash-pruned-285B (this repo)	285 B MoE, 23 B active	170.17 GB	80.6	82.3	80.8	28.3	97.7%	932
Ripple-65B-Preview (BF16, 2 cards)	65 B dense	129.57 GB	89.0	90.5	86.2	56.7	99.4%	751
Ripple-65B-Preview NVFP4 (Ricursive)	65 B dense	57.22 GB	88.9	90.4	85.4	58.3	99.4%	758
Ripple-65B-Preview FP8 (this repo)	65 B dense	65.85 GB	89.5	91.0	85.4	61.7	99.3%	756
Ripple-65B-Preview NVFP4A16	65 B dense	37.92 GB	88.6	90.3	84.6	56.7	99.4%	832

overall pools the two suites as 1,370 items, weighted 85.4% knowledge and 14.6% tool calling by item count. Read it with finished: overall scores an answer that overran the token budget as wrong, and cannot say whether the model needed the room or failed to stop. A gap under 1.0 in overall is a tie. Sizes are as each card reports them, which mixes GB and GiB.

What's in it

Ripple-65B-Preview is a dense 65 B model (80 layers, hidden size 5120, 80 query and 8 KV heads, MLP width 40,960).

	params	share
MLP gate + up + down, 80 layers	50.33 B	77.6%
attention q + k + g + v + o, 80 layers (`g` is the output gate)	13.42 B	20.7%
embeddings and LM head (untied)	1.03 B	1.6%

modules	count	format
every decoder Linear (q, k, g, v, o, gate, up, down)	640	FP8 E4M3, one scale per output channel; activations FP8, one scale per token
embeddings, LM head, norms, Q/K norms, value-residual scalars	all	BF16, byte-identical to the release

The checkpoint is compressed-tensors. vLLM fuses q/k/g into one module and gate/up into another, so each fused group shares one format.

Licence

Apache-2.0, the base model's licence. This is a modified version of ricursiveAI/Ripple-65B-Preview: the decoder's linear layers were quantized and config.json carries a quantization_config. Everything else is unchanged.

_{primitive ·
more models ·
inference economics for production LLM systems}