Gemma 4 E2B-it QAT, FP8 W8A8 with int4 embeddings, text only (unofficial)
Google's quantization-aware-trained weights, text model only, with every Linear layer stored as
FP8 E4M3 (one float32 scale per output channel) and activations quantized to FP8 per token at
run time (compressed-tensors float-quantized). Embeddings and lm_head are the int4 tables of the -emb4 build, unchanged; lm_head is untied.
| FP8 Linear modules | 276 |
| Quantized values | 1,876,819,968 |
| Relative RMS error against the QAT weights | 2.64 % |
| Largest error, as a fraction of its row's largest value | 3.57 % |
| Checkpoint | 3.40 GiB |
QAT trained the weights onto a 4-bit grid with one scale per group of 32; FP8 with one scale per
output channel cannot represent those per-group scales, so this build rounds the QAT weights
again, and it also quantizes activations. Built with fp8_text.py (in this repo) build-on; no calibration data.
Limitations
- Text only.
- Unofficial. Report problems here, not to Google.