Gemma 4 31B-it QAT, exact Q4_0 GGUF, text only (unofficial)
This is an unofficial build, made and published independently of Google. It is a llama.cpp GGUF of Gemma 4 31B-it in which every 4-bit weight is the value Google's quantization-aware training (QAT) produced, the token embedding included. It is 17.29 GB, against 17.65 GB for Google's gemma-4-31B-it-qat-q4_0-gguf. The E4B counterpart, built by the same script and measured, is xbill9/gemma-4-E4B-it-qat-q4_0-exact-gguf.
| Google Q4_0 GGUF | This GGUF | |
|---|---|---|
| File size | 17,651,001,568 B | 17,287,669,984 B |
token_embd |
Q6_K | Q4_0 |
| Rebuilt values bit-identical to the QAT source | — | 96.81 % |
What is in it
- Language model only. Vision lives in Google's separate mmproj GGUF, which is not included here.
- Metadata copied byte for byte from Google's
gemma-4-31B_q4_0-it.ggufat revision59dde24: tokenizer, chat template, hyperparameters and tensor order. - Weights rebuilt from
google/gemma-4-31B-it-qat-q4_0-unquantizedat revision1e4d8be, as Q4_0:
| Tensor | Count | Google's type | Values bit-identical to the source |
|---|---|---|---|
token_embd.weight |
1 | Q6_K | 96.82 % |
attn_k |
60 | Q4_0 | 96.81 % |
attn_output |
60 | Q4_0 | 96.69 % |
attn_q |
60 | Q4_0 | 96.81 % |
attn_v |
50 | Q4_0 | 96.84 % |
ffn_down |
60 | Q4_0 | 96.84 % |
ffn_gate |
60 | Q4_0 | 96.80 % |
ffn_up |
60 | Q4_0 | 96.81 % |
- Copied unchanged: the norms and scale vectors (F32).
Every rebuilt block landed on a 4-bit grid; values that are not bit-identical differ by the fp16 rounding of the
block scale. Google's GGUF uses llama.cpp's standard step (the block's largest magnitude divided by 8); this build
recovers each block's trained step instead (divided by 8, 7, … 1, whichever puts every value on an integer level,
refined by least squares). evidence/build_report.json has the per-tensor counts.
Status: built and checked offline against its source; not yet loaded in llama.cpp. Divergence from bf16, speed and memory were measured for the E4B build only.
Rebuilding it
hf download google/gemma-4-31B-it-qat-q4_0-unquantized --revision 1e4d8be --local-dir src
hf download google/gemma-4-31B-it-qat-q4_0-gguf gemma-4-31B_q4_0-it.gguf --revision 59dde24 --local-dir gguf
python3 gguf_exact.py src gguf/gemma-4-31B_q4_0-it.gguf gemma-4-31B-it-q4_0-exact.gguf
Needs only numpy. The script stops if any block fails to land on a 4-bit grid.
Limitations
- Text only.
- Not yet run in llama.cpp.
- Unofficial. Report problems here, not to Google.
License and attribution
Gemma 4 is released by Google DeepMind under the Apache 2.0 license. This repository redistributes Google's weights in a changed storage format, under the same license. The weights, training and the original model card (kept unchanged in ORIGINAL_README.md) are Google DeepMind's; this build and card are not affiliated with or endorsed by Google.