LFM3-1.7B NVFP4 (Schur Replay)
Modified weights notice: this repo contains a quantized (NVFP4) derivative of LiquidAI/LFM3-1.7B. The feed-forward weights were re-quantized by BennyDaBall; everything else is Liquid AI's original BF16. Not affiliated with or endorsed by Liquid AI.
Same layer map, same file format and same speed as Liquid's official LFM3-1.7B-NVFP4. Better 4-bit weights: 1.39x closer to BF16 under W4A4, 1.93x on the weights alone. For llama.cpp / LM Studio, see the GGUF twin: BennyDaBall/LFM3-1.7B-NVFP4-GGUF (1.28 GB, 2.03x closer to BF16 than a converted Liquid NVFP4, runs on stock llama.cpp).
Weights updated 2026-11-04. The feed-forward weights were re-solved with a post-pass added to my Schur Replay solver: coordinate descent on the NVFP4 codes against the GPTQ objective, then an end-state re-pick of the E4M3 block scales, run twice. Same format, same layer map, same input scales, same speed. KLD to BF16 under W4A4 went from 0.0461 to 0.0451 (paired bootstrap over the eval set, 95% interval excludes zero), with every eval category improving. Liquid's official NVFP4 measures 0.0628 on the same eval, so this build now sits 28% closer to the BF16 model than the vendor file, at the same size and the same speed.
What changed vs Liquid's NVFP4
| Liquid NVFP4 | This repo | |
|---|---|---|
| Layer map | NVFP4 W4A4 on language feed-forward weights + activations; attention, convs, embeddings, head, norms BF16 | Identical (Liquid's map, cited from their card) |
| Weight solver | Round-to-nearest (ModelOpt default; I checked: weight error identical to plain RTN) | GPTQ + Schur Replay block-scale selection + Four Over Six, then a code and block-scale refinement post-pass |
| Activations | Static per-tensor input_scale, dynamic per-16 block scales | Same scheme, recalibrated on my calibration set |
| Format | ModelOpt NVFP4 safetensors | Same (drop-in) |
Schur Replay (arXiv 2609.36654): inside the GPTQ loop, each 16-weight block tries a window of E4M3 scale codes plus the Four Over Six map-to-4 scale (arXiv 2512.02010). Every candidate replays the block's GPTQ updates and is scored by its real GPTQ loss (the Schur complement of the not-yet-quantized columns). Each row keeps its best. After the solver, a post-pass walks the codes column by column (coordinate descent on the same GPTQ quadratic, two neighbouring grid points per weight, two sweeps) and then re-picks each 16-weight block's E4M3 scale with the codes fixed, twice over; every accepted move lowers the loss the solver minimized. Hessians come from 599k calibration tokens (tool calls, chat, reasoning, raw text). The chat, tool-call and reasoning answers are the BF16 model's own generations.
KLD vs BF16 (lower is better)
Gold = LiquidAI/LFM3-1.7B BF16 (transformers). Held-out eval: 150 sequences, 124,790 scored tokens (every model-written span: tool calls, chat, reasoning, plus raw text). NVFP4 math reproduced exactly as the W4A4 kernels compute it.
| W4A4 (vLLM / SGLang path) | Mean KLD | Median | p99 | Top-1 agree | Tool | Chat | Reasoning | Raw |
|---|---|---|---|---|---|---|---|---|
| Liquid NVFP4 | 0.0628 | 0.0187 | 0.584 | 90.3% | 0.0353 | 0.0707 | 0.0378 | 0.0870 |
| This repo | 0.0451 | 0.0131 | 0.426 | 91.8% | 0.0231 | 0.0504 | 0.0254 | 0.0658 |
| Weight-only view (W4A16) | Mean KLD | Top-1 agree |
|---|---|---|
| Liquid NVFP4 | 0.0332 | 92.9% |
| This repo | 0.0172 | 95.0% |
Speed and memory (one RTX 5090, same sitting)
Same kernels (CUTLASS SM120 block-scaled FP4 GEMMs, 48 per step), same 1.86 GiB of weights. Single stream, 512 tokens, vLLM 0.32.0 + LFM3 PR overlay, KV pool 1 GiB:
| Checkpoint | Decode tok/s | Prefill tok/s (2.3k prompt) | Weights | Peak VRAM |
|---|---|---|---|---|
| BF16 | 290 | 33,555 | 3.36 GiB | 5,424 MiB |
| Liquid NVFP4 | 379 | 54,685 | 1.86 GiB | 3,945 MiB |
| This repo | 402 | 53,368 | 1.86 GiB | 4,001 MiB |
SGLang (PR 46858): BF16 345, Liquid NVFP4 513, this repo 457 tok/s decode. The gap vs Liquid flips sign between runtimes (FlashInfer/CUTLASS autotuner tile picks); summed GPU kernel time differs by ~2%, so treat the two NVFP4 files as the same speed.
Usage
Recommended sampling (Liquid's card): temperature 0.1, top_k 50, repetition_penalty 1.1 (math/STEM: temperature 1.0, top_p 0.95, top_k 20). Thinking is on by default; tool calls use Liquid's XML format.
LFM3 is not in a vLLM or SGLang release yet (Nov 1, 2026). It works today with vLLM v0.32.0 + vllm-project/vllm PR 64959, or SGLang nightly + sgl-project/sglang PR 46858, exactly like Liquid's NVFP4:
vllm serve BennyDaBall/LFM3-1.7B-NVFP4 --trust-remote-code
python -m sglang.launch_server --model-path BennyDaBall/LFM3-1.7B-NVFP4 --reasoning-parser auto --tool-call-parser auto
License
LFM Open License v1.0 (copy in LICENSE, unchanged). Key terms, quoted from it:
"You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications" (Section 4)
"The rights granted under this License for Commercial Use are conditioned upon You or Your Legal Entity not exceeding the Threshold." (Section 5(a)), where the Threshold is "annual revenue of 10 million United States dollars ($10,000,000) or more." (Section 1)
Free for any use, commercial included, while you (and your corporate group) make under $10M a year; above that, commercial use needs a license from Liquid AI.
Credits
- Model: Liquid AI, LFM3-1.7B. NVFP4 layer map from Liquid's own LFM3-1.7B-NVFP4.
- Methods: GPTQ (Frantar et al. 2022), Four Over Six (Cook et al., arXiv 2512.02010), Schur Replay (arXiv 2609.36654); the post-pass follows the coordinate-descent refinement idea of CDQuant and ReQuant (arXiv 2608.07019), re-implemented on the NVFP4 grid with block-scale re-picking.
- Quantization: BennyDaBall.