Gemma 4 26B-A4B-it QAT, FP8 E4M3 W8A8 with int4 embeddings, text only (unofficial)
This is an unofficial build, made and published independently of Google. It holds Google's quantization-aware-trained (QAT) Gemma 4 26B-A4B-it weights, from google/gemma-4-26B-A4B-it-qat-q4_0-unquantized (revision f1e06dc), text model only, with every Linear layer stored as FP8 E4M3 (one float32 scale per output channel) and activations quantized to FP8 per token at run time (compressed-tensors float-quantized). Embeddings and lm_head are the int4 tables of xbill9/gemma-4-26B-A4B-it-qat-q4_0-w4a16-ct-text-emb4 (revision 38971d3), copied unchanged; lm_head is untied.
The source stores each layer's 128 experts as two fused banks (experts.gate_up_proj, experts.down_proj). This build splits them into one module per expert, experts.{i}.{gate,up,down}_proj, the layout of the W4A16 repack xbill9/gemma-4-26B-A4B-it-qat-q4_0-w4a16-ct-text. The router (router.proj) stays bf16.
Measured against the QAT weights (verify_report.json) |
|
|---|---|
| FP8 Linear modules | 11,725 |
| Quantized values | 24,483,430,400 |
| Relative RMS error | 2.60 % |
| Largest error, as a fraction of its row's largest value | 3.57 % |
| Other tensors, byte-identical to their source | 391 of 391 |
| Checkpoint | 23.66 GiB |
QAT trained the weights onto a 4-bit grid with one scale per group of 32 values. FP8 with one scale per
output channel cannot represent those per-group scales, so this build rounds the QAT weights again, and it
also quantizes activations. For the exact QAT grid use xbill9/gemma-4-26B-A4B-it-qat-q4_0-w4a16-ct-text.
Status: built and checked offline against its source; not served. The AMD MI300X sweep at this size ran bf16, q4w4a16, fp8 only.
Built with
fp8_text.py (in this repo, build-on the int4-embedding build),
which imports helpers from repack_q4_0.py (also here). No calibration data is used.
Limitations
- Text only.
- Not served or evaluated.
- Unofficial. Report problems here, not to Google.
License and attribution
Gemma 4 is released by Google DeepMind under the Apache 2.0 license. This repository redistributes Google's weights in a changed storage format, under the same license. The weights, training and the original model card (kept unchanged in ORIGINAL_README.md) are Google DeepMind's; this build and card are not affiliated with or endorsed by Google.