Gemma 4 12B — closest Q4_0 to BF16

30% lower KL than Google's official Q4_0 QAT and 2.2× closer to BF16 than Unsloth's imatrix Q4_0. Same size as Google. Standard Q4_0 runtime.

Q4_0	KL to BF16 ↓	Top-1 agreement ↑
QUASAR	0.048	94.3%
Google QAT	0.069	93.5%
Unsloth Q4_0 (imatrix)	0.108	91.6%
Standard Q4_0 (llama-quantize, no imatrix)	0.156	89.8%

The W4A16 twin shows the same pattern under transformers: KL 0.034 to BF16 vs 0.044 for Google's W4A16 QAT (top-1 95.5% vs 94.8%).

ollama run hf.co/QUASAR-QAT/gemma-4-12B-it-QUASAR-Q4_0-GGUF
llama-cli -hf QUASAR-QAT/gemma-4-12B-it-QUASAR-Q4_0-GGUF

Better fidelity than the popular alternatives

QUASAR also gets lower KL than Unsloth's Q4_0 repack of Google's QAT checkpoint, and than LM Studio's Q4_K_M — a more expressive format:

GGUF	KL to BF16 ↓
QUASAR Q4_0	0.048
Unsloth Google-QAT repack (Q4_0)	0.053
LM Studio Q4_K_M	0.110

Same BF16 reference, prompts, llama.cpp build, and backend.

Full evaluation and reproduction details → EVAL.md

Downstream benchmarks

Higher point estimates on all four downstream benchmarks than Google's QAT model (measured on the W4A16 twin, which carries the same trained codes).

Benchmark	QUASAR	Google QAT
MMLU	70.4	67.8
IFEval	87.6	87.3
ARC-Challenge	51.3	50.1
AGIEval	24.2	22.6
Average	58.4	57.0

The MMLU gain is statistically robust: +2.6 points, 95% CI [+2.0, +3.2] (paired bootstrap over 14,042 questions). Full configs, BF16 results and CIs are in EVAL.md.

Native QAT GGUF

Native Q4_0 QAT, not a post-training re-quantization. The GGUF stores QUASAR's trained INT4 codes and scales directly; all 328 quantized tensors are verified against the W4A16 twin (native_q4_0_receipt.json, verifier in scripts/). No re-rounding and no custom runtime.

Multimodal support is included through the BF16 vision projector from google/gemma-4-12B-it.

Run it

Ollama: ollama run hf.co/QUASAR-QAT/gemma-4-12B-it-QUASAR-Q4_0-GGUF — the vision projector is picked up automatically.
LM Studio: search QUASAR-QAT gemma-4-12B in Discover.
llama.cpp, vision: llama-mtmd-cli -hf QUASAR-QAT/gemma-4-12B-it-QUASAR-Q4_0-GGUF --image photo.jpg -p "Describe this image." --jinja
Thinking: on by default in the embedded Gemma 4 chat template; toggle with --chat-template-kwargs '{"enable_thinking": false}' (llama-cli / llama-server).
Sampling: Gemma 4 defaults — temperature 1.0, top-k 64, top-p 0.95.
Context: 262,144 tokens.

What's inside

File	Size
`gemma-4-12B-it-QUASAR-Q4_0.gguf` — Q4_0 linears (trained group-64 scales), Q6_K token embeddings, chat template embedded	6.98 GB
`mmproj-gemma-4-12B-it-BF16.gguf` — unmodified BF16 vision projector from `google/gemma-4-12B-it`	175 MB
`native_q4_0_receipt.json`, `SHA256SUMS`, `EVAL.md`, `scripts/` (packer/verifier, eval harvest, paired bootstrap)

QUASAR

QUASAR is loss-aware quantization-aware training. It improves the low-bit reconstruction during training, then exports to standard deployment formats with zero inference overhead.

This checkpoint was healed against the BF16 teacher for ~600M tokens: one epoch over ~377k prompts self-distilled from BF16 Gemma 4 12B-it with thinking on, fp32 master weights, INT4 group-64.

W4A16 compressed-tensors (vLLM) twin of this checkpoint: QUASAR-QAT/gemma-4-12B-it-QUASAR-W4A16-G64
Gemma 4 E4B, same recipe: GGUF · W4A16 · collection · Gemma 4 E4B & 12B — QUASAR 4-bit QAT
Base model: google/gemma-4-12B-it · Paper: arXiv:2608.13966 · Org: huggingface.co/QUASAR-QAT

License

Based on Gemma 4 by Google DeepMind. Upstream LICENSE/NOTICE apply.

Citation

arxiv.org/abs/2608.13966

@article{counathe2026quasar,
  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},
  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},
  journal={arXiv preprint arXiv:2608.13966},
  year={2026}
}

QUASAR-QAT/gemma-4-12B-it-QUASAR-Q4_0-GGUF