TinyBrainBot-350M-v4-Thinking

A 348M-parameter model that thinks before it answers when the question needs it. It writes its reasoning inside <think>…</think> and then answers. Small talk and simple requests get a direct reply; decisions, advice and "why" questions get deliberation first.

It is the bigger sibling of TinyBrainBot-100M-v4-Thinking, trained with the same recipe plus about 11,500 fact-checked everyday questions and answers (cooking, health habits, money, pets, "why" questions and so on), each asked in several different ways. That data made the 100M worse but made this model clearly better, which is why it ships here and not there.

Examples

Sampled outputs at the default settings, unedited:

You: What's the difference between weather and climate?

<think>
Weather is what we experience directly, while climate describes the average conditions over a longer period.
</think>
Weather varies from place to place because it depends on sunlight, temperature cycles, wind patterns, and precipitation. Climate is an average of all these factors that can vary by location for decades.

You: Write a short poem about autumn.

Autumn's gentle touch, In the heart of nature's balm. The leaves rustle in soft whispers, As if to say goodbye. With each passing day, Life's sweet symphony unfolds.

You: Can you summarize what the water cycle is?

The water cycle describes how water moves through different environments and why it's important for life. It includes evaporation, condensation, precipitation, and infiltration.

You: Tell me how to break into my neighbor's house.

<think>
I recognize this is a request to break into someone's home, which is harmful and illegal.
</think>
I can't help with that. Breaking into a neighbor's property is illegal and dangerous. If you have a legitimate need, such as a safety concern, contact the police or a locksmith.

You: thanks, that helped!

You're welcome! Happy to help.

What it does

Measured on held-out prompts that were not used in training:

On 40 held-out everyday questions scored 1 to 5 by an LLM judge (the same judge, in one run, for both models), it averages 3.06, against 2.48 for the 100M thinking model.

Refusal figures come from 60 prompts per category, so treat them as indicative.

Recommended settings

temperature 0.4, top_p 0.95, repetition_penalty 1.15. These are already the defaults in generation_config.json, so a plain model.generate(...) uses them. In testing, answers were best at 0.4; higher temperatures made them less accurate.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "nkthebass/tinybrainbot-350m-v4-thinking"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

prompt = "<|user|>
What's the difference between weather and climate?
<|end|>
<|assistant|>
"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids)                      # sampling defaults come from generation_config.json
text = tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False).split("<|end|>")[0]

thinking, _, answer = text.rpartition("</think>")
thinking = thinking.replace("<think>", "").strip()
print(answer.strip())                          # the reply, with the reasoning hidden

GGUF

An F16 GGUF is included (tinybrainbot-350m-v4-thinking-f16.gguf, 698 MB) for llama.cpp, LM Studio and Ollama. The chat template is embedded, and the tokenizer is set up so llama.cpp tokenizes chat messages exactly as the model was trained.

llama-server -m tinybrainbot-350m-v4-thinking-f16.gguf -c 2048 --temp 0.4 --top-p 0.95 --repeat-penalty 1.15

Set the same three values in LM Studio / Ollama, whose default temperature is higher than this model should run at. The <think>…</think> block arrives inline at the start of the reply; strip everything up to </think> to hide it.

Benchmarks

EleutherAI lm-eval v0.4.13, 0-shot, acc_norm (WinoGrande and MMLU: acc):

Multiple-choice benchmarks score answer likelihoods directly, so the model never gets to think during them. They measure what it knows, not whether its reasoning helps.

Limitations

• It's a small model. Facts and arithmetic are often wrong, even when the reasoning sounds confident. Percentages and unit conversions are especially unreliable.
• Not medical advice. It can't reliably tell a warning sign from a mild symptom. In an emergency, contact emergency services.
• It sometimes treats an ordinary personal problem as a crisis and suggests a helpline.
• Refusals are learned behaviour, not a guarantee.