Gemma 4 E2B compact for Fluent Books (V23d, IQ4_XS GGUF)
A small on-device model for Fluent Books, an English reader for Russian speakers, built for devices with 3 GB of memory (an iPad mini 5, an iPhone XS) where our full model (A2, 1.35 GB) does not fit. 778,390,784 bytes (IQ4_XS), SHA-256 e96d962672cd23c276fd9d4a8ad9c237506ee312c477e59c50acb7507303e82b.
It is not a general chat model. It answers the short questions the app asks about a word in the sentence being read: which of the dictionary's Russian meanings fits, whether an expression is used with one of its meanings, and a short explanation ("why here?").
How it was made
- A2 is our LoRA fine-tune of
google/gemma-4-E2B-itwith its vocabulary cut from 262,144 to 40,000 tokens. - The feed-forward layers of A2's 20 wide layers (the ones that share their K/V cache) were narrowed from 12,288 to 6,144 neurons (the ones kept have the largest mean |activation| at the input of
down_projtimes the norm of theirdown_projcolumn, measured on training rows), and the 6 least important of those layers were taken out (29 of 35 layers remain). Layer importance = the KL divergence to the full model when the layer is skipped. - The pruned model was distilled from A2 (cross-entropy on the answers plus KL to A2's logits) in several rounds on our training data, the last one on dataset v7-slim.
- Quantized to IQ4_XS with an importance matrix made from the training prompts.
What it does, measured (our own sets, 4–5 Nov 2026; the file as the app runs it, the app's prompts)
| V23d | A2 (1.35 GB) | |
|---|---|---|
| Choice of the meaning of a word, 462 held-out uses (of ours) | 390 | 386 |
| Held-out dictionary words (1,942 uses) | 1,543 | 1,540 |
| "Is the expression used here?" (AUC, 807 labelled uses) | 0.858 | 0.860 |
| Speed on an iPad mini 5 (CPU), a meaning, warm; V23d with the app's compact question | 3.6 s | 11–22 s (the full question, cold) |
| Memory in the app | ≈ 350 MB | — |
What it does not do well
- Translation of sentences. On our blind-judged sets (two judges, a translation counts only if both mark it right) it is below A2 and below Apple's translator: 105 of 120 traps (gender, "you" to several people, names that are words) against A2's 114–115; expressions 68–72 of 100 against A2's 82–86. The app asks Apple's translator first for sentences when this model is in use.
- Idioms and names that are also words. Halving the feed-forward width costs the expressions; training on 170,000 more rows from larger Gemma teachers and on-policy distillation did not recover them (we tried both).
- It sometimes falls into English in the middle of a Russian word. The app forbids tokens of plain Latin letters for its Russian answers; without such a ban expect it.
- Our evaluation sets are small and ours; the judges were language-model agents.
Training data
- Sentences from public-domain novels (Project Gutenberg), translated by Claude Sonnet 5.5 agents; 59,932 more such sentences translated by A2.
- Aligned pairs of public-domain human translations (ru.wikisource.org, Project Gutenberg).
- Dictionary-meaning answers on which Gemma 4 26B-A4B and 31B agreed with a probability of 0.9 or more.
- Rows built from licensed learner's dictionaries owned by the authors: example sentences matched to word senses, and the Russian equivalents of English proverbs. The dictionaries' text is not distributed here. A check of 300 trained proverbs: 12 (4 %) come out word for word, the rest in other words.
- No source sentence comes from a textbook or from our evaluation sets.
Use
llama.cpp (a build that knows Gemma 4, b11208 or newer). The prompts are the app's (ModelPrompts.swift: a question of the sense of a word with the numbered Russian meanings and an answer opened as {"ru":"); the model is meant to be called by the app, not chatted with.
Base model: Google's Gemma 4 E2B (Apache-2.0); thanks to the Gemma team.