Gemma 4 31B-it QAT, int8 W8A8, text only (unofficial)
This is an unofficial build, made and published independently of Google. It holds Google's quantization-aware-trained (QAT) Gemma 4 31B-it weights, from google/gemma-4-31B-it-qat-q4_0-unquantized (revision 1e4d8be), text model only, stored as compressed-tensors int8 W8A8: int8 weights with one bf16 scale per output channel, activations quantized to int8 per token at run time (format: int-quantized). The QAT weights already sit on the Q4_0 grid; this build rounds them to int8 per channel.
| Layer | Tensors | int8 against QAT, relative error |
|---|---|---|
mlp.down_proj |
60 | 0.93 %–1.18 % (mean 1.02 %) |
mlp.gate_proj |
60 | 0.80 %–0.97 % (mean 0.87 %) |
mlp.up_proj |
60 | 0.80 %–0.99 % (mean 0.85 %) |
self_attn.k_proj |
60 | 0.83 %–1.35 % (mean 0.97 %) |
self_attn.o_proj |
60 | 0.82 %–1.16 % (mean 0.98 %) |
self_attn.q_proj |
60 | 0.83 %–1.07 % (mean 0.92 %) |
self_attn.v_proj |
50 | 0.77 %–1.17 % (mean 0.93 %) |
Checkpoint: 29.91 GiB.
Status: built and checked offline against its source; not served. The AMD MI300X sweep at this size ran bf16, q4w4a16, fp8 only.
Built with
w8a8_from_qat.py (in this repo, with the repack_q4_0.py helpers they import).
No calibration data is used.
Limitations
- Text only.
- Not served or evaluated.
- Unofficial. Report problems here, not to Google.
License and attribution
Gemma 4 is released by Google DeepMind under the Apache 2.0 license. This repository redistributes Google's weights in a changed storage format, under the same license. The weights, training and the original model card (kept unchanged in ORIGINAL_README.md) are Google DeepMind's; this build and card are not affiliated with or endorsed by Google.