Gemma 4 26B-A4B-it QAT, exact Q4_0 GGUF, text only (unofficial)
This is an unofficial build, made and published independently of Google. It is a llama.cpp GGUF of Gemma 4 26B-A4B-it in which every 4-bit weight is the value Google's quantization-aware training (QAT) produced, the token embedding included. It is 14.25 GB, against 14.44 GB for Google's gemma-4-26B-A4B-it-qat-q4_0-gguf. The E4B counterpart, built by the same script and measured, is xbill9/gemma-4-E4B-it-qat-q4_0-exact-gguf.
| Google Q4_0 GGUF | This GGUF | |
|---|---|---|
| File size | 14,439,363,584 B | 14,249,047,040 B |
token_embd |
Q6_K | Q4_0 |
| Rebuilt values bit-identical to the QAT source | — | 97.11 % |
What is in it
- Language model only. Vision lives in Google's separate mmproj GGUF, which is not included here.
- Metadata copied byte for byte from Google's
gemma-4-26B_q4_0-it.ggufat revisiond1c082b: tokenizer, chat template, hyperparameters and tensor order. - Weights rebuilt from
google/gemma-4-26B-A4B-it-qat-q4_0-unquantizedat revisionf1e06dc, as Q4_0:
| Tensor | Count | Google's type | Values bit-identical to the source |
|---|---|---|---|
token_embd.weight |
1 | Q6_K | 96.66 % |
attn_k |
30 | Q4_0 | 96.79 % |
attn_output |
30 | Q4_0 | 96.77 % |
attn_q |
30 | Q4_0 | 96.77 % |
attn_v |
25 | Q4_0 | 96.83 % |
ffn_down |
30 | Q4_0 | 97.54 % |
ffn_down_exps |
30 | Q4_0 | 97.13 % |
ffn_gate |
30 | Q4_0 | 97.43 % |
ffn_gate_up_exps |
30 | Q4_0 | 97.14 % |
ffn_up |
30 | Q4_0 | 97.33 % |
The experts stay in Google's fused layout, ffn_gate_up_exps and ffn_down_exps, one bank per layer, each rebuilt row for row from the source's fused bank (experts.gate_up_proj, experts.down_proj). ffn_down_exps.scale and the other F32 tensors are copied unchanged.
- Copied unchanged: the norms and scale vectors (F32).
Every rebuilt block landed on a 4-bit grid; values that are not bit-identical differ by the fp16 rounding of the
block scale. Google's GGUF uses llama.cpp's standard step (the block's largest magnitude divided by 8); this build
recovers each block's trained step instead (divided by 8, 7, … 1, whichever puts every value on an integer level,
refined by least squares). evidence/build_report.json has the per-tensor counts.
Status: built and checked offline against its source; not yet loaded in llama.cpp. Divergence from bf16, speed and memory were measured for the E4B build only.
Rebuilding it
hf download google/gemma-4-26B-A4B-it-qat-q4_0-unquantized --revision f1e06dc --local-dir src
hf download google/gemma-4-26B-A4B-it-qat-q4_0-gguf gemma-4-26B_q4_0-it.gguf --revision d1c082b --local-dir gguf
python3 gguf_exact.py src gguf/gemma-4-26B_q4_0-it.gguf gemma-4-26B-A4B-it-q4_0-exact.gguf
Needs only numpy. The script stops if any block fails to land on a 4-bit grid.
Limitations
- Text only.
- Not yet run in llama.cpp.
- Unofficial. Report problems here, not to Google.
License and attribution
Gemma 4 is released by Google DeepMind under the Apache 2.0 license. This repository redistributes Google's weights in a changed storage format, under the same license. The weights, training and the original model card (kept unchanged in ORIGINAL_README.md) are Google DeepMind's; this build and card are not affiliated with or endorsed by Google.