Gemma 4 31B-it QAT, FP8 E4M3 W8A8 with int4 embeddings, text only (unofficial)
This is an unofficial build, made and published independently of Google. It holds Google's quantization-aware-trained (QAT) Gemma 4 31B-it weights, from google/gemma-4-31B-it-qat-q4_0-unquantized (revision 1e4d8be), text model only, with every Linear layer stored as FP8 E4M3 (one float32 scale per output channel) and activations quantized to FP8 per token at run time (compressed-tensors float-quantized). Embeddings and lm_head are the int4 tables of xbill9/gemma-4-31B-it-qat-q4_0-w4a16-ct-text-emb4 (revision c8e1d82), copied unchanged; lm_head is untied.
Measured against the QAT weights (verify_report.json) |
|
|---|---|
| FP8 Linear modules | 410 |
| Quantized values | 29,286,727,680 |
| Relative RMS error | 2.64 % |
| Largest error, as a fraction of its row's largest value | 3.57 % |
| Other tensors, byte-identical to their source | 421 of 421 |
| Checkpoint | 28.77 GiB |
QAT trained the weights onto a 4-bit grid with one scale per group of 32 values. FP8 with one scale per
output channel cannot represent those per-group scales, so this build rounds the QAT weights again, and it
also quantizes activations. For the exact QAT grid use xbill9/gemma-4-31B-it-qat-q4_0-w4a16-ct-text.
Status: built and checked offline against its source; not served. The AMD MI300X sweep at this size ran bf16, q4w4a16, fp8 only.
Built with
fp8_text.py (in this repo, build-on the int4-embedding build),
which imports helpers from repack_q4_0.py (also here). No calibration data is used.
Limitations
- Text only.
- Not served or evaluated.
- Unofficial. Report problems here, not to Google.
License and attribution
Gemma 4 is released by Google DeepMind under the Apache 2.0 license. This repository redistributes Google's weights in a changed storage format, under the same license. The weights, training and the original model card (kept unchanged in ORIGINAL_README.md) are Google DeepMind's; this build and card are not affiliated with or endorsed by Google.