naderu-mini-4b

A compact on-device offline-chat model by Naderu — a BytesBrains Pte. Ltd. venture.

naderu-mini is the on-device fallback that powers the Naderu assistant inside the AiToolK.it app when the device is offline — no cloud, no data leaving the phone. It runs locally via llama.cpp. The assistant presents as Naderu (the offline version); the model's name is naderu-mini, and it states its upstream base plainly — in this card and in its own answers.

Provenance

Base: Qwen/Qwen3-4B (Apache-2.0), pinned revision 1cfa9a72.
Method: identity/behaviour SFT (LoRA r=8, 16 layers, lr 3e-5, 600 iters via MLX, non-thinking mode), fused, then quantized to GGUF Q6_K.
License: Apache-2.0 (inherits the base's terms).

What it's for

The offline AI-chat mode in BytesBrains Flutter apps: rewrite/shorten/tone-shift, summarize pasted text, prompt-smithing, brainstorming, everyday Q&A — mobile-length, answer-first. It is honest about being offline: it declines connected tasks (URLs, live data) and suggests full Naderu when back online, and it never claims to take actions in the app.

Out of scope: factual reliability without verification, professional (medical/legal/ financial) advice, code beyond trivial snippets, long-document work, tool/RAG, and anything needing live data or guaranteed correctness.

Evaluation

Passed all 8 gates of the Naderu behaviour suite v1 (held-out probes), verified on this Q6_K GGUF: identity_leakage 0, identity_rate 0.929, deferral 1.00, boundary 1.00, task 1.00, action_claims 0, language-routing 1.00, refusal 1.00. Note: the smaller Q4_K_M/Q5_K_M quants fail the identity gate (0.857) after quantization, which is why Q6_K is the released build.

Usage (llama.cpp)

Send this exact system prompt (the model was trained on it) and use non-thinking mode:

You are Naderu — the offline, on-device version of the Naderu assistant, running inside the
AiToolK.it app. You have no internet access: no browsing, live data, or cloud tools, and you
cannot take actions in the app. Be warm, concise, and practical: answer first, keep replies
short for a phone screen. If a task needs connectivity or current information, say so briefly
and suggest trying full Naderu when back online. If you are not sure of a fact, say you're not
certain rather than guessing.

llama-cli -hf bytesbrains/naderu-mini-4b --jinja -sys "<system prompt above>" -p "Who are you?"

Needs ~4.5 GB free RAM. Verify the download by sha256 (Q6_K: e5fbf8f4…).

Built by Naderu — we train foundation models into specialised ones, release them with model cards and clear licensing, and back every claim with a real evaluation.