primitive-ai/LFM2.5-24B-A2B-NVFP4A16

4-bit expert weights, 16-bit everywhere else: closer to BF16 than Liquid's own FP8, at 60% of its size.

NVFP4 weight-only build of LiquidAI/LFM2.5-24B-A2B at 14.95 GB, on one GPU.
The routed experts store NVFP4 weights and compute in 16-bit (W4A16). The short-convolution blocks, attention, the two dense layers, embeddings and routers stay BF16.

Accuracy suites cannot separate these builds. On our 1,370 items every build in the table below, BF16 included, scores within 0.7 points of every other. So the table also scores each build against BF16's own output: 400 BF16 reasoning traces, the extra negative log-likelihood each quant assigns them per token, and how often its most likely next token is BF16's. That measurement is deterministic, and on it the builds range from +8.6 to +66.8.

---

Why this quant

• 🎯 Closest to BF16 of the builds we measured. +8.6×10⁻³ nats per token above BF16 on BF16's own traces, with top-1 agreement 96.1%. Liquid's FP8 release scores +10.0 and 96.0%; the published W4A4 NVFP4 builds score +20.6 to +66.8.
• 🗜️ 14.95 GB against 47.94 GB in BF16 and 24.75 GB for the FP8 release. On one 96 GB card that leaves a 3,682,347-token KV pool at 32K context, against 2,077,736 for BF16.
• ⚡ 218 tok/s single-stream and 1,395 at concurrency 32, against 204 and 1,131 for the FP8 release and 179 and 861 for BF16. The W4A4 builds run 1,639 to 1,923 tok/s at 32: their expert activations are 4-bit too, which is faster on FP4 tensor cores. On the same expert weights we measured that switch at +20.1 instead of +8.6 in drift from BF16.
• 📏 Level with BF16 on the accuracy suites. 83.9 overall against BF16's 83.9; knowledge 85.4 against 85.6, tool calling 74.8 against 74.4.
• 🧩 Stock vLLM 0.31.0, no patches. compressed-tensors, nvfp4-pack-quantized, weights only.

---

Serve it

hf download primitive-ai/LFM2.5-24B-A2B-NVFP4A16 --local-dir ./LFM2.5-24B-A2B-NVFP4A16

docker run --gpus all --ipc=host -p 8000:8000 -v $PWD:/models \
  vllm/vllm-openai:v0.31.0 \
  --model /models/LFM2.5-24B-A2B-NVFP4A16 \
  --max-model-len 32768 --gpu-memory-utilization 0.90 \
  --enable-auto-tool-choice --tool-call-parser lfm2 --reasoning-parser qwen3

The 0.31.0 image (sha256:c1c9f6fd5c109ba7f0546a59f5b2f15fb87f64c77782e90a27b648b42a8e67c3) serves Lfm2MoeForCausalLM as released, and the experts load on vLLM's Marlin NVFP4 kernel. Boot takes about 172 s on an idle card. The model always thinks before it answers, so give agentic turns a generous max_tokens. Liquid recommends temperature 0.1, top_k 50, repetition_penalty 1.05; our measurements use the fixed protocol below for every build instead, so they compare builds with each other, not with Liquid's published numbers.

---

Measured

One RTX PRO 6000 Blackwell, 96 GB, one card, no tensor parallelism. The 1,170-item knowledge suite and the 200-item tool-calling suite, temperature 0.6 / top_p 0.95 / top_k 20, thinking on, a 16,384-token budget, concurrency 32, auto-scored with no LLM judge, BF16 KV cache for every row except the one marked †. Throughput is 8K in / 512 out, prefix-cache free, two seeds per cell, every build benched on the same card one after another with the second GPU idle.

Drift and agreement come from 400 BF16 traces of the knowledge suite (521,362 completion tokens) scored by each build in one deterministic pass; repeating a pass gives the same number to six decimal places, and the standard error over traces is 0.3 to 1.2. Tool-calling is the mean of 5 to 10 runs per build, with a within-build standard deviation of 0.5 to 2.1, so that column is one band. Knowledge is two runs per build. † served with the vLLM loader patch from its own repository, as its card instructs; its config also sets an FP8 KV cache, which vLLM applies, so that row runs FP8 KV and has the larger pool.

What this table does not cover:

• Languages other than English. The model supports sixteen; both suites are English.
• Context past 32K. The model's window is 131,072 tokens; we did not test the 4-bit experts there.
• Other GPUs. Only the RTX PRO 6000 Blackwell (sm_120) was tested.

---

Comparable with our other models

Accuracy numbers move for reasons that have nothing to do with the model: a shorter token budget, a different temperature, or whether the model was allowed to reason at all. So every number in this table, on this card and on our other cards, comes from the one fixed protocol described above, the same 1,370 items, auto-scored, no LLM judge.

overall pools the two suites as 1,370 items, weighted 85.4% knowledge and 14.6% tool calling by item count. Read it with finished: overall scores an answer that overran the token budget as wrong, and cannot say whether the model needed the room or failed to stop. A gap under 1.0 in overall is a tie. Sizes are as each card reports them, which mixes GB and GiB.

---

What's quantized to what

LFM2.5-24B-A2B is 23.97 B parameters, 2.45 B active per token, and 95.7% of them are routed experts.

compressed-tensors, format nvfp4-pack-quantized, one config group with input_activations: null. w1 and w3 of each expert share one global scale, so the fused projection vLLM builds keeps the exact values. The expert weights are calibrated, not rounded to nearest.

---

Licence

LFM Open License v1.0, the base model's licence; a copy is in this repository. This is a modified version of LiquidAI/LFM2.5-24B-A2B: the routed-expert weights were quantized to NVFP4 and config.json carries a quantization_config. All other weights are unchanged, and the model and its copyright belong to Liquid AI, Inc. Commercial use is licensed only for legal entities below US$10 million in annual revenue (section 5 of the licence).

---

_{primitive · more models · inference economics for production LLM systems}