Qwen3.5-2B for Fluent Books (v3, Q4_K_M GGUF)
A LoRA fine-tune of Qwen/Qwen3.5-2B, merged and quantised to Q4_K_M (with an importance matrix), for the on-device assistant of the Fluent Books reading app (an English reader for Russian speakers). It runs on llama.cpp; the app downloads this single file. It follows v2 (same size, same prompts).
- File:
qwen3.5-2b-ft-v3.Q4_K_M.gguf, 1,312,164,544 bytes, SHA-256fef8dfbb9894a7e31c75cfc1b958b6ef62e5840f9fd4ffb6ce3ebe9bc5e469e0. - Licence: Apache License 2.0, as the base model (see
LICENSE). Copyright of the base model: Alibaba Cloud. - Prompt format: ChatML with an empty thinking block after the assistant tag (
<think>\n\n</think>\n\n), exactly as the app sends it. The model is trained on the app's own prompts and works best with them: English→Russian sentence translation, choosing the dictionary meaning of a word in its sentence (JSON{"ru": "...", "sense": N}), a short Russian explanation of a word or of a grammar point in its sentence, the subject and main verb of a long sentence, and a yes/no check ("0"/"1") whether an English expression is used in a sentence.
Training
LoRA (rank 32, every linear layer, scale 20), 2 epochs (12,484 steps), learning rate 1e-5, batch 8, loss on the answer only; MLX on a Mac. 49,936 training examples:
| part | examples | where the answers come from |
|---|---|---|
| ordinary sentences (EN→RU) | 13,752 | sentences from public-domain novels (Project Gutenberg); translations written by Claude Sonnet 5.5 agents |
| hard cases: gender, «you» to several people, names that are words | 1,974 / 1,644 / 3,466 | the same |
| expressions, real and false (including contexts of about 680 expressions and phrasal verbs added for v3) | 7,352 / 1,982 | the same, with a hint to the translating agent; whether the words are the expression: two Claude agents agree |
| translation traps by kind: passive, modal perfect, inversion, cleft, participle clause, gerund subject, idioms, false friends | 2,160 | the same, with a hint per kind |
| aligned sentence pairs of public-domain human translations | 6,292 | Twain/Engelhardt (Tom Sawyer, Huckleberry Finn, Tom Sawyer Abroad and Detective: 2,874), Swift/Frankovsky (974), Wells/Pimenova (864) and Wells/Morozov (553), Haggard/Bekertova (587), London/Vershinina (440); texts from ru.wikisource.org, English from Project Gutenberg, aligned with LaBSE, kept at confidence 0.70 or more |
| dictionary-meaning choice | 7,851 | answers on which Gemma 4 26B-A4B and 31B (Apache-2.0) agree with a probability of 0.9 or more |
| word explanation, grammar-point explanation, sentence skeleton | 962 / 728 / 590 | Claude Sonnet 5.5 agents |
| expression check (0/1) | 1,183 | the digit on which Gemma 4 26B-A4B and 31B agree |
No source sentence comes from a textbook or from the evaluation sets (checked by exact match and by 8-word overlap). «The Picture of Dorian Gray», the held-out books of the expression test set and the sentences of every gold set are excluded. Translation rows that two independent judges rejected were removed (about 980 for v2, and 1,421 more for v3, of which 1,092 were human pairs rejected for loose alignment or free rendering).
Evaluation (blind judges, our own sets; 2 Nov 2026)
| v2 | this model | Gemma 4 E2B | Apple Translation | |
|---|---|---|---|---|
| 223 extended traps (passive, cleft, gerund subject, idioms …), share of checks passed, two judges, one blind set | 68.4 % | 74.4 % | 77.1 % | 69.7 % |
| chrF++ against a human translation of 1,000 sentences of one novel (not in the training data) | 35.3 | 35.8 | 36.3 | 36.7 |
| dictionary-meaning choice, of 462 | 360 | 377 | 360 | – |
| expression translation, of 100 | 78 | 78 | 81 | 76 |
| pairwise with Apple on 138 sentences (model : Apple : equal) | 40 : 44 : 54 | 46 : 46 : 46 | 43 : 24 : 71 | – |
| trap sentences (gender, «you», names), of 120 | 105 | 103 (gender 36, «you» 39, names 28) | 105 | 111 |
The figures of the first rows were judged in one session on one blind set; the last two rows are from earlier and later sessions with different judges: do not read small differences (one or two sentences of 120) as a change. "Has not forgotten the other tasks": judges prefer this model's word and grammar explanations to the base model's (41 : 5 and 33 : 1); the sentence skeleton is found as well as by the base model.
Known limitations
- Names that are also ordinary words (Hope, Pluto, Slightly) are still translated worse than by Apple's translator (28 of 40 on our set; the target was 33), and on sentences with «you» it is weaker than Apple's.
- Latin letters sometimes stay in a translation (13 of 1,000 sentences; Apple 1).
- The 0/1 expression check separates right from wrong uses better than the base model (AUC 0.81 against 0.65 for the base and 0.78 for v2), but its probability of "1" is shifted upwards: use a threshold of about 0.70, not 0.15 (balanced accuracy 0.74 on our 865 labelled uses; the curve is flat between 0.60 and 0.80).
- Evaluated on English→Russian novel sentences only; not for other languages, long documents or safety-critical use. The evaluation sets and judges are our own; most judges are language models.