Gemma 4 12B-it QAT, compressed-tensors int8 W8A8, text only (unofficial)
Google's quantization-aware-trained weights, text model only, stored as int8 W8A8: int8 weights
with one bf16 scale per output channel, activations quantized to int8 per token at run time
(format: int-quantized), the same scheme as an llm-compressor W8A8 export. The source is the QAT
weights, which already sit on the Q4_0 grid, rounded to int8 per channel. Embeddings and norms are
bf16; the vision and audio towers are dropped. The E2B twin is
xbill9/gemma-4-E2B-it-qat-w8a8-int8.
| Layer | Tensors | int8 against QAT, relative error |
|---|---|---|
mlp.down_proj |
48 | 0.78–1.36 % (mean 1.03 %) |
mlp.gate_proj |
48 | 0.80–1.20 % (mean 0.85 %) |
mlp.up_proj |
48 | 0.78–1.13 % (mean 0.85 %) |
self_attn.k_proj |
48 | 0.83–1.59 % (mean 0.94 %) |
self_attn.o_proj |
48 | 0.79–1.45 % (mean 0.91 %) |
self_attn.q_proj |
48 | 0.78–1.22 % (mean 0.89 %) |
self_attn.v_proj |
40 | 0.79–1.32 % (mean 0.89 %) |
Checkpoint: 12.03 GiB. Made with
w8a8_from_qat.py.
Limitations
- Text only.
- Unofficial. Report problems here, not to Google.