MNN-Pocket-0.3B-RU-Chat GGUF

A Russian instruction-following assistant in 0.36 B parameters, converted to GGUF for llama.cpp. It answers instead of continuing. 50 M tokens of SFT on top of the continuation model produced that.

Weights, data and evaluation live in Rahspide/MNN-Pocket-0.3B-RU-Chat.

Run

The ChatML template is inside the file, so --jinja is enough:

llama-cli -hf Rahspide/MNN-Pocket-0.3B-RU-Chat-GGUF:Q4_K_M --jinja \
  -p "Объясни, что такое инфляция" -n 60

Without the template the model continues text instead of answering and behaves like its sister.

Quants

Measured on a 16-thread x86 CPU: ~123 tok/s generation on Q4_K_M, ~102 tok/s on Q8_0.