Muse Glimmer 30B — native NVFP4 W4A4 for Blackwell, 21.8 GiB

Blackwell W4A4 build of our QUASAR-trained Muse Glimmer 30B: 20% lower KL to BF16 than Red Hat's NVFP4 W4A4 checkpoint and 45% lower than RTN, with 93.2% top-1 agreement. All 416 decoder projections use NVFP4. Standard vLLM / compressed-tensors, no custom kernels.

NVFP4 / vLLM · W4A4 / Blackwell (this repo) · GGUF / llama.cpp · Muse Glimmer collection · QUASAR paper

With the same 4-bit activations but BF16 weights, KL is 0.0434; quantizing the weights with QUASAR adds only 0.0096 KL. KL / top-1: per-token forward KL(BF16 ‖ model) and top-1 agreement on 948 held-out agentic prompts of the BF16 model's own responses (1.19M response tokens), under a vLLM-verbatim simulation of the W4A4 NVFP4 activation path (per-token group-16 E2M1, dynamic FP8 local scales, static per-layer global scale) — same harness and prompts as the W4A16 card. Raw results in eval/.

Run it

pip install "vllm>=0.28"

vllm serve QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4 \
  --tool-call-parser muse_glimmer --reasoning-parser muse_glimmer --enable-auto-tool-choice \
  --max-model-len 131072

compressed-tensors nvfp4-pack-quantized with NVFP4 input_activations. On Blackwell (B200 / GB200 / RTX 5090), vLLM uses the native NVFP4 W4A4 path. For Hopper, use the dedicated W4A16 build.

Capability benchmarks: for GPQA-Diamond, MMLU-Pro, AIME'25, coding, tool use and RULER through 128K on the underlying QUASAR-trained weights, see the W4A16 reference build.

Technical details

All 416 decoder projections use NVFP4 E2M1 weights with group size 16 and FP8-E4M3 group scales. Embeddings, LM head, norms and vision tower remain BF16. This repo uses the same QUASAR-trained weights as the W4A16 checkpoint and adds calibrated NVFP4 activation scales for Blackwell W4A4 execution.

Related

• W4A16 / Hopper (the reference build): QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4
• llama.cpp / Ollama (native Q4_0 QAT): QUASAR-QAT/Muse-Glimmer-30B-QUASAR-Q4_0-GGUF
• Collection: Muse Glimmer 30B — QUASAR 4-bit QAT · Base model: meta-models/Muse-Glimmer-30B · Paper: arXiv:2608.13966
• Running this checkpoint? Share benchmark results, deployment notes, or integrations in Discussions.

Citation

@article{counathe2026quasar,
  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},
  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},
  journal={arXiv preprint arXiv:2608.13966},
  year={2026}
}