Rahspide/MNN-Pocket-0.3B-RU-Chat

Status: v1 released. Instruction-tuned for two epochs over 24.8 M unique tokens (18.8 M Russian + 6 M English replay), 50 M tokens consumed in total, on one Kaggle T4 in ~6.5 h. Weights and GGUF builds are in this repository.

Russian instruction-following assistant, built by SFT on top of Rahspide/MNN-Pocket-0.3B-RU. It answers instead of continuing; that dialogue shape is what 50 M tokens of SFT bought.

Two models, two jobs

The two tasks reward opposite behaviour. Trained on both jobs, one checkpoint does each badly: it answers when it should continue, and continues when it should answer.

Why start from the Russian continuation model

The base MNN-Pocket-0.3B is bilingual zh/en and has essentially no Russian; a direct SFT of the official instruct model would fight both problems at once (new language and new format). Starting from the continuation model means Russian is already paid for, and SFT only has to teach the dialogue shape.

Recipe (as run)

• Start point: Rahspide/MNN-Pocket-0.3B-RU (step 36,621).
• Chat template: the model's own ChatML (<|im_start|> / <|im_end|>), untouched.
• Loss only on assistant tokens; system and user turns are masked context.
• Multi-turn dialogue kept as one example; packing with explicit boundaries.
• One run at 50 M tokens (1,525 optimizer steps, 32 k tokens each), checkpoints every 300 steps.
• 85 / 15 Russian-to-English sampling per micro-batch; the English share keeps bilingual instruction following.
• lr 1e-4, cosine, 5% warmup, full fine-tuning, fp16 autocast on T4 with fp32 master weights.
• ~6.5 h on one Kaggle T4.

Data (as used)

Filters applied to saiga_scored: Russian only, dropped rows flagged is_bad_by_regex, 80–6,000 characters per dialogue, 2–8 turns. Roles bot / gpt are normalised to assistant; empty turns are dropped.

Quality spot-check

Greedy generation, Q4_K_M, CPU:

The model answers instead of continuing the prompt. That is all this table proves.

Builds

llama-cli -hf Rahspide/MNN-Pocket-0.3B-RU-Chat:Q4_K_M --jinja -p "Объясни, что такое инфляция" -n 60

The MNN build runs on MNN's runtime (llm_demo, built with -DMNN_BUILD_LLM=ON). Runtime sources: alibaba/MNN, the LLM export and inference guide, and the official Android app apps/Android/MnnLlmChat. Its llm_config.json keeps the ChatML template, so it answers out of the box:

cd mnn/MNN-Pocket-0.3B-RU-Chat-INT4 && llm_demo config.json prompt.txt

165 MB on disk, ~497 tok/s prefill and ~80 tok/s decode on 12 CPU threads. The answer is coherent Russian; the INT4 weights cost some precision against Q8_0, as expected.

~123 tok/s generation on Q4_K_M, ~102 tok/s on Q8_0 (16-thread x86 CPU, no GPU). Apply the model's own ChatML template (--jinja in llama.cpp, apply_chat_template in transformers). Without it the model continues text instead of answering and behaves like its sister.

Reproduce

# chat smoke test through the model's own template
python bench/chat_smoke.py --model Rahspide/MNN-Pocket-0.3B-RU-Chat --cpu

# throughput of the quantised build
LLAMA_CLI=/path/to/llama-cli bash bench/bench_cpu.sh gguf/MNN-Pocket-0.3B-RU-Chat-Q4_K_M.gguf --chat

bench/chat_smoke.py applies the chat template from tokenizer_config.json, generates greedily and asserts that the answer does not echo the prompt, which is a cheap check that the template and the checkpoint still match. bench/bench_cpu.sh --chat wraps the prompt in ChatML before timing, so the numbers are comparable with the table above.

Evaluation

60 held-out Russian prompts from the saiga_scored validation slice, greedy decoding, the model's own template:

The sheet with all 60 prompts and answers is in review/blind_sheet.md.

A comparison against taobao-mnn/MNN-Pocket-0.3B was attempted on the same prompts and dropped. That checkpoint degenerates on Russian input, repeating the prompt in bold markdown fragments:

Q: Как стать популярным в яндекс дзен?
A: Конечный откорт: **"Популярный в яндекс дзен"** — это **не** в яндекс (в яндекс) — **в яндекс** — **в яндекс** — …

So it is not a baseline.

Limitations

• 0.36 B parameters: short answers, weak facts, no reliable multi-step reasoning.
• Russian SFT data is mostly machine-translated or model-generated; style is flatter than human chat.
• The sister's tokenizer cost applies here too: about 3.3 tokens per Russian word, so ~3× more decode steps per Russian sentence than for English. The vocabulary is byte-level, so rare and uppercase Cyrillic letters can end up split across tokens; a streaming decoder has to buffer partial UTF-8 sequences.
• Not a coding model, not a tool-calling model, no long-context dialogue.
• English answers are thinner than Russian ones. Replay keeps the ability alive; it does not polish it.

Releases

Licence and attribution

Weights: Apache-2.0, inherited through MNN-Pocket-0.3B-RU from taobao-mnn/MNN-Pocket-0.3B-Base.

@misc{mnn_pocket_03b,
  title  = {MNN-Pocket-0.3B},
  author = {Taobao MNN team},
  year   = {2026},
  url    = {https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B}
}