TinyBrainBot-360M-A75M-v5-Instruct

One small model for chat, instructions, reasoning and math. A 383M-parameter mixture-of-experts model (24 experts per layer, 4 active, so 75M parameters work per token), trained from scratch. It talks, follows instructions, works through arithmetic and word problems in a visible <think> scratchpad, and remembers what you told it earlier in the conversation.

Try it in your browser: the Q8_0 GGUF runs locally on your device, and nothing you type is sent anywhere.

Scores

IFEval was run with lm-eval and the chat template, greedy, with the <think> block removed before scoring. The Index uses the full HellaSwag / ARC / PIQA sets plus ArithMark-3, 0-shot.

Math in chat (greedy, exact final answer):

Safety (held-out prompts): it declines 93% of harmful requests and wrongly refuses 8.3% of benign ones.

Quick start

from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "nkthebass/tinybrainbot-360m-a75m-v5-instruct"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
msgs = [{"role": "user", "content": "Leo has 30 pens. Mia has 9 fewer pens than Leo. How many pens does Mia have?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=200)        # defaults from generation_config.json
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

GGUF (llama.cpp, LM Studio, Jan, Ollama): tinybrainbot-360m-a75m-v5-instruct-Q8_0.gguf (409 MB) or -F16.gguf (768 MB). The chat template is embedded, and both tokenize exactly like the transformers version.

llama-cli -m tinybrainbot-360m-a75m-v5-instruct-Q8_0.gguf -cnv --temp 0.4 --top-p 0.9 --repeat-penalty 1.15 --logit-bias 241-inf --logit-bias 255-inf

Recommended sampling: temperature 0.4, top-p 0.9, repetition penalty 1.15. The two --logit-bias flags block the byte tokens that start every emoji: the tokenizer has no emoji pieces, so the model would otherwise spell them out byte by byte.

<think> blocks are plain text, not special tokens. Strip them yourself if you only want the answer:

import re; answer = re.sub(r"<think>.*?</think>", "", reply, flags=re.S).strip()

What it does well (and doesn't)

• Math: column arithmetic with every carry and borrow written out, and word problems worked in a <think> scratchpad ("'9 fewer than' means subtraction...").
• Conversation: it remembers what you said ("You said you play basketball"), holds its ground when it is right ("I see why, but I'm sure. The capital of Australia is Canberra"), and stays supportive on personal topics.
• Not yet: it thinks before answering on math, but not on open "why/should" questions; there it answers directly.
• Word order matters in relation problems. It is reliable when the known amount comes first ("Leo has 30 pens. Mia has 9 fewer"), but can take the numbers in the wrong order when the relation comes first ("Mia has 9 fewer pens than Leo. Leo has 30").
• At 75M active parameters it still makes up facts.

Unedited sample conversations, including mistakes, are in conversation_samples.md.

Training

1. Pretraining. TinyBrainBot-360M-A75M-v4.1-MoE-Base: ~18.2B tokens, plus a factual-Q&A expert patch.
2. Arithmetic decay (0.5B tokens). The learning rate re-warms and decays on a mix of 45% generated grade-school arithmetic (our own programs, every number computed in Python), 40% Q&A, facts, Cosmopedia and Wikipedia, and 15% Cosmopedia WikiHow.
3. SFT (0.39B tokens), one mix for every skill (training them in separate stages forgot what each later stage didn't repeat):

4. DPO. 10,383 preference pairs (UltraFeedback, instruction-constraint pairs checked by code, and judge-ranked pairs), beta 0.5, LR 1e-6, 2 epochs.

No benchmark training or test data was used. The GSM8K-derived sets from our earlier math model were left out on purpose.

Limitations

• English only. It confabulates facts, so check anything important.
• The tokenizer collapses newlines and extra spaces, so it can't write code or keep text layout.
• 2,048-token context.