Qwen3.8-27B NVFP4, trained with QUASAR

Project page · Code

19.7 GB · 496/496 transformer linears in NVFP4 (W4A4) · 0.909 GPQA-Diamond vs. 0.914 BF16 · serves directly with vLLM.

Among the public NVFP4 builds compared below, QUASAR is both the smallest and the highest-scoring: 90.9 GPQA-Diamond and 100% AIME'26.

Independent evaluation: A third-party NVFP4 shootout on the official Qwen3.8-27B repo compares QUASAR against other public NVFP4 checkpoints under the same evaluation and serving setup.

Smaller sibling — Qwen3.5-4B: W4A16 / vLLM · W4A4 / Blackwell · Q4_0 GGUF / llama.cpp · collection · paper

More QUASAR checkpoints: Gemma 4 collection · Muse-Glimmer collection · paper.

QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 is a 4-bit NVFP4 version of Qwen/Qwen3.8-27B, produced with QUASAR, a quantization-aware training (QAT) method.

QUASAR trains the NVFP4 weights directly against the frozen BF16 model, then exports standard NVFP4 weights with no custom inference path. This lets us quantize all 496 transformer linears—including attention and gated delta-net—to NVFP4 while preserving near-BF16 quality: the smallest of the compared public NVFP4 builds, while scoring highest among them on both GPQA-Diamond and AIME'26.

📄 Paper: QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction

How to run

Compatible with vLLM, with no conversion step:

pip install "vllm>=0.27"

vllm serve QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.85 \
  --max-num-seqs 512 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}' \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3

On a 32 GB card such as an RTX 5090, lower the context to --max-model-len 65536.

Requires an NVIDIA GPU with FP4 support (Blackwell, compute capability 10.0+).

Quality and size comparison

Two other public NVFP4 builds of this model, evaluated under the same setup (GPQA-Diamond: 2 runs, n=396; AIME'26: 3 repeats, n=90). Bold = best among the NVFP4 builds.

Training

One epoch of loss-aware NVFP4 quantization-aware distillation against the frozen BF16 teacher: global batch size 32, learning rate 1e-6, 2446 steps.

Citation

arxiv.org/abs/2608.13966

@article{counathe2026quasar,
  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},
  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},
  journal={arXiv preprint arXiv:2608.13966},
  year={2026}
}