Gemma 4 E2B: English-only vocabulary (en50v), GGUF Q4_0
This is Gemma 4 E2B-it with its vocabulary cut from 262,144 to 87,830 tokens, keeping English, code and common symbols. No training or fine-tuning was done. The transformer blocks are bit-identical to ggml-org/gemma-4-E2B-it-GGUF (Q4_0). Only the per-token tables (token embeddings, per-layer token embeddings, output layer) and the tokenizer were trimmed, row by row on the quantized bytes.
Method, code and full results: github.com/Nilanjan112233/llm-vocab-trimmer
| Original | This model | Change | |
|---|---|---|---|
| File size | 2.84 GB | 1.67 GB | −41% |
| Peak RAM (llama-server) | 3.02 GB | 2.53 GB | −16% |
| Time per tool-calling task (with MTP draft) | 2.00 s | 1.34 s | −33% |
| Generation speed | 14.5 tok/s | 21.5 tok/s | +48% |
| Tool-calling accuracy (30 tasks, JSON schema) | 100% | 100% | same |
| English perplexity (WikiText-103 test, 40×512) | 48.1 ± 1.9 | 45.7 ± 1.8 | within error |
Measured on CPU only (Intel i5-11300H, 4 threads), llama.cpp b12360, temperature 0. The slightly lower perplexity mostly comes from probability being shared among fewer tokens. Read it as "no measurable loss", not as an improvement.
Files
| File | What |
|---|---|
gemma-4-E2B-en50v-Q4_0.gguf |
main model |
mtp-gemma-4-E2B-en50v-Q4_0.gguf |
MTP draft for speculative decoding, trimmed with the same list. It only works with the file above. |
keep_ids_50v.json |
the original token ids that were kept, in order |
Run
llama-server -m gemma-4-E2B-en50v-Q4_0.gguf \
--spec-type draft-mtp -md mtp-gemma-4-E2B-en50v-Q4_0.gguf \
-t 4 -c 4096 --reasoning off
You need a llama.cpp build with Gemma 4 and MTP support. I tested b12360.
Limits
- English and code only. Other languages still work through byte fallback, but they take more tokens and quality drops. For example, a Hindi sentence goes from 12 to 29 tokens.
- Text only. I haven't tested image or audio input.
- Reasoning ability is the same as the base model. The gains are size and speed.
- My tool-calling benchmark is small (30 tasks). It is not a general quality evaluation.
License and credits
This is a modified version of Gemma 4 E2B by Google, released under the Apache-2.0 license. The changes are the vocabulary trim described above. The base GGUF and MTP draft come from ggml-org.