Gemma 4 26B-A4B-it QAT, int8 W8A8, text only (unofficial)
This is an unofficial build, made and published independently of Google. It holds Google's quantization-aware-trained (QAT) Gemma 4 26B-A4B-it weights, from google/gemma-4-26B-A4B-it-qat-q4_0-unquantized (revision f1e06dc), text model only, stored as compressed-tensors int8 W8A8: int8 weights with one bf16 scale per output channel, activations quantized to int8 per token at run time (format: int-quantized). The QAT weights already sit on the Q4_0 grid; this build rounds them to int8 per channel.
The source stores each layer's 128 experts as two fused banks (experts.gate_up_proj, experts.down_proj). This build splits them into one module per expert, experts.{i}.{gate,up,down}_proj, the layout of the W4A16 repack xbill9/gemma-4-26B-A4B-it-qat-q4_0-w4a16-ct-text. The router (router.proj) stays bf16.
| Layer | Tensors | int8 against QAT, relative error |
|---|---|---|
experts.down_proj |
3,840 | 0.66 %–0.76 % (mean 0.69 %) |
experts.gate_proj |
3,840 | 0.69 %–0.98 % (mean 0.75 %) |
experts.up_proj |
3,840 | 0.56 %–0.93 % (mean 0.75 %) |
mlp.down_proj |
30 | 0.79 %–0.86 % (mean 0.82 %) |
mlp.gate_proj |
30 | 0.76 %–0.83 % (mean 0.80 %) |
mlp.up_proj |
30 | 0.76 %–0.83 % (mean 0.81 %) |
self_attn.k_proj |
30 | 0.81 %–1.29 % (mean 0.94 %) |
self_attn.o_proj |
30 | 0.84 %–1.38 % (mean 0.96 %) |
self_attn.q_proj |
30 | 0.82 %–1.12 % (mean 0.87 %) |
self_attn.v_proj |
25 | 0.80 %–1.15 % (mean 0.92 %) |
Checkpoint: 24.23 GiB.
Status: built and checked offline against its source; not served. The AMD MI300X sweep at this size ran bf16, q4w4a16, fp8 only.
Built with
w8a8_from_qat.py (in this repo, with the repack_q4_0.py helpers they import).
No calibration data is used.
Limitations
- Text only.
- Not served or evaluated.
- Unofficial. Report problems here, not to Google.
License and attribution
Gemma 4 is released by Google DeepMind under the Apache 2.0 license. This repository redistributes Google's weights in a changed storage format, under the same license. The weights, training and the original model card (kept unchanged in ORIGINAL_README.md) are Google DeepMind's; this build and card are not affiliated with or endorsed by Google.