Gemma 4 E2B-it QAT, W4A16 with int4 per-layer embeddings, text only (unofficial)
The text-only W4A16 repack xbill9/gemma-4-E2B-it-qat-q4_0-w4a16-ct-text with its per-layer embedding table packed as int4. embed_tokens and lm_head stay bf16 and tied. It exists to separate the per-layer table's effect from lm_head's: compare it with the -emb4 build, which also packs embed_tokens and an untied lm_head.
QAT put the embedding tables on the same 4-bit grid as the linears (group 32), so the packing
recovers that grid; an off-grid group would abort the build.
| Table | bf16 | int4 |
|---|---|---|
embed_tokens_per_layer |
4.375 GiB bf16 | 1.230 GiB int4 + f16 scales |
embed_tokens_per_layer: 0 off-grid groups; 74.26% of values bit-identical, worst error 6.58e-03 of the group max; scales 0.00301..0.91.
Checkpoint: 2.96 GiB. Linear layers are unchanged from the W4A16 repack.
Made with embed_int4.py
(default fp16 scales). Needs vLLM 0.29 or later
(CompressedTensorsEmbeddingWNA16Int).
Limitations
- Text only.
- Unofficial. Report problems here, not to Google.