Muse Glimmer 30B — native NVFP4 QAT, 22 GiB

The most faithful 4-bit Muse Glimmer: 72% lower KL to BF16 than Red Hat's NVFP4 build and 39% lower than NVIDIA's, at the smallest size of the three, while quantizing all 416 decoder projections to NVFP4. Within 0.3 points of BF16 across eight standard benchmarks, at or above BF16 on GPQA-Diamond, MMLU-Pro and AIME'25, and the only 4-bit build above BF16 at 32K, 64K and 128K RULER. Standard vLLM NVFP4, no custom kernels.

NVFP4 / vLLM (this repo) · W4A4 / Blackwell · GGUF / llama.cpp · Muse Glimmer collection · QUASAR paper

KL / top-1: per-token forward KL(BF16 ‖ model) and top-1 agreement on 948 held-out agentic prompts of the BF16 model's own responses (1.19M response tokens), one harness for every row, each quantized checkpoint decoded to BF16 from the artifact it ships as. *Red Hat's checkpoint ships with 4-bit activations and is measured as shipped (W4A4). avg8 = arc_c / arc_e / hellaswag / winogrande / truthfulqa / mmlu / gsm8k / ifeval. RULER = 5 tasks × 100 samples.

Run it

pip install "vllm>=0.28"

vllm serve QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4 \
  --tool-call-parser muse_glimmer --reasoning-parser muse_glimmer --enable-auto-tool-choice \
  --max-model-len 131072

Loads straight from the Hub (compressed-tensors nvfp4-pack-quantized, weight-only / W4A16). Validated on H100 with vLLM 0.28 (Marlin W4A16 kernel): chat, reasoning channel, tool calls through the muse_glimmer parsers, 128K context.

Blackwell / W4A4: use the W4A4 sibling, which adds calibrated activation scales for native NVFP4 W4A4 kernels.

Selected benchmarks

Full same-harness results, including coding, tool-use and agentic evaluations, are available in eval/.

Technical details

All 416 decoder projections in the 52 layers (q / k / v / o, the attention output gate, MLP gate / up / down) are NVFP4: E2M1 weights in groups of 16 with FP8-E4M3 group scales under one fp32 tensor scale; token embeddings, LM head, norms and the vision tower stay BF16, bit-identical to the original. The weights were trained with QUASAR quantization-aware training by self-distillation from the frozen BF16 model on 846M tokens of its own agentic, reasoning and tool-use responses — no new data, a drop-in lower-precision replacement. The served artifact is exactly the trained quantized model: the packed NVFP4 weights replay bit-exactly to the evaluated weights (1,572,732,928 / 1,572,732,928 groups, 0 flipped elements — export_ct_report.json), and fused q/k/v and gate/up groups share their tensor scale by construction, so vLLM's fused-layer loading changes nothing.

Related

• Blackwell / W4A4 build of this checkpoint: QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4
• llama.cpp / Ollama (native Q4_0 QAT): QUASAR-QAT/Muse-Glimmer-30B-QUASAR-Q4_0-GGUF
• Collection: Muse Glimmer 30B — QUASAR 4-bit QAT · Base model: meta-models/Muse-Glimmer-30B · Paper: arXiv:2608.13966 · Org: huggingface.co/QUASAR-QAT
• Running this checkpoint? Share benchmark results, deployment notes, or integrations in Discussions.

Citation

arxiv.org/abs/2608.13966

@article{counathe2026quasar,
  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},
  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},
  journal={arXiv preprint arXiv:2608.13966},
  year={2026}
}