KW5-230M Instruct

A Kiswahili assistant small enough to run on a laptop CPU. It was trained by Regnant from the tokenizer up, on Kiswahili text, and fine-tuned to answer from what you give it: a passage, an earlier turn in the conversation, a sum to work.

230,423,424 parameters, embeddings tied. Instruction-tuned from KW5-230M, the third-generation base, pretrained on 7.90B tokens of Kiswahili.

What it is measurably good at, on a frozen suite of 1,464 Kiswahili items built before any v3 training:

• Arithmetic. 91.4% on bare sums up to four digits and 70.5% on mixed expressions with brackets, working each step in a <mawazo> block. The second-generation instruct model scored 2.3% on the first.
• Remembering a conversation. 98.8% recall of a fact told 3 to 9 turns earlier, and 82.5% "you never told me" when it was not.
• Reading a passage. 58.4% on 250 questions over real Swahili Wikipedia and TyDiQA passages, and 70.0% when the passage does not hold the answer and the right reply is to say so.

What it gets wrong is in its own section below, with numbers. Read it before you trust an answer.

Two things to get right

1. Encode with the encoder this repo ships. kw5_tokenizer.py encodes line by line, so the first word of every line keeps its word-boundary piece. That is how the corpus was packed and how the model was fine-tuned. Plain sp.encode on the whole prompt fragments the first word after every newline (\nEleza becomes ▁ <0x0A> E leza), and the model has never seen that.

2. Start with id 1 and stop at <|end|> (id 7). <s> is a SentencePiece control symbol: encoding the string "<s>" gives three ordinary pieces, not id 1. Prepend the id yourself. Generation ends at <|end|>, not </s>.

The training format:

<s><|user|>
{your message}<|end|>
<|assistant|>

A conversation repeats the user and assistant blocks, each closed by <|end|>. A <|system|> block may open it.

Quick start (runs in Google Colab as-is)

!pip install -q huggingface_hub sentencepiece safetensors

import sys, torch, sentencepiece as spm
from huggingface_hub import snapshot_download

path = snapshot_download("regnant-io/kw5-230M-instruct")    # ~1.04 GB, cached
sys.path.insert(0, path)          # the repo ships modeling_kw5v2.py and kw5_tokenizer.py
from modeling_kw5v2 import KW5V2ForCausalLM
from kw5_tokenizer import make_encoder

device = "cuda" if torch.cuda.is_available() else "cpu"
model = KW5V2ForCausalLM.from_pretrained(path).to(device)
sp = spm.SentencePieceProcessor(model_file=f"{path}/tokenizer.model")
encode = make_encoder(sp)

BOS, END, EOS = 1, 7, 2

def chat(turns, max_new_tokens=256):
    """turns: [("user", "..."), ("assistant", "..."), ("user", "...")]"""
    ids = [BOS]                                  # the id, never encode("<s>")
    for role, text in turns:
        if role == "user":
            ids += encode(f"<|user|>\n{text}<|end|>\n")
        else:
            ids += encode("<|assistant|>\n") + encode(text) + encode("<|end|>\n")
    ids += encode("<|assistant|>\n")
    out = model.generate(ids, max_new_tokens=max_new_tokens, temperature=0.0, eos_id=END)
    return sp.decode([t for t in out[len(ids):] if t not in (END, EOS)])

print(chat([("user", "Hesabu: 71 - 17 + 20 = ?")]))
print(chat([("user", "Mji mkuu wa Kenya ni upi?"),
            ("assistant", "Mji mkuu wa Kenya ni Nairobi."),
            ("user", "Na wa Uganda je?")]))

Every number on this card was measured greedy (temperature=0.0), as above.

A stock AutoModelForCausalLM will not load it: the block has Canon layers, short causal convolutions that no Llama-family model has. modeling_kw5v2.py needs only PyTorch and safetensors.

What it sounds like

These are replies from the evaluation run, unedited.

A sum, worked. Only arithmetic carries a <mawazo> block, so a plain question is answered without reasoning in front of it:

Hesabu: 71 - 17 + 20 = ?

<mawazo>Tunafuata mpangilio wa hesabu: mabano kwanza, kisha kugawanya na
kuzidisha kutoka kushoto kwenda kulia, kisha kujumlisha na kutoa kutoka
kushoto kwenda kulia.
Tunatoa 71 - 17. ... Kwa hiyo 71 - 17 = 54. Sasa tuna: 54 + 20.
Tunajumlisha 54 + 20. ... Kwa hiyo 54 + 20 = 74.</mawazo>
Jibu ni 74.

A passage. Asked how many people live in Morogoro region, over a passage that says it: 2,218,492.

A follow-up. After "Mji mkuu wa Kenya ni upi?" and its answer, "Na wa Uganda je?" gets Mji mkuu wa Uganda ni Kampala. The second-generation model refused this one and named Kampala inside the refusal.

Something it was never told. Asked about a city several turns into a conversation that never mentioned one: Hujaniambia mji wowote. ("You haven't told me any city.")

Who made it. Every self-introduction in its fine-tuning names it KW5, an AI assistant from Regnant, across sixteen ways of asking.

Release bars

Fourteen bars were frozen on 2026-10-15, the day before training began, so they could not drift toward whatever the model scored. A missed bar is reported as missed. Seven of fourteen are met.

Wilson intervals. The followup row has 8 items and is indicative only.

Beside other small models

The same frozen suite, greedy, each model in its own chat template, on one Colab T4 (2026-10-28). Percent correct.

Read the last two rows with care. The conversation and refusal axes (recall, never-told, state, persistence, follow-ups, declining) are scored against the replies a Kiswahili assistant should give: "Hujaniambia ...", "Kifungu hakisemi hilo.", "Sijui ...". KW5 was fine-tuned on declines in that voice, and the other models were not. swahili-gemma-1b answered "Mama yangu anaitwa nani?" by asking which mother was meant, a sensible reply that this scorer counts as a miss. The Qwen models often answered in English. On the 1,116 content items, where the reply is a number, a span or a choice, KW5 leads a model four times its size by 4.7 points, and loses word problems and AfriMGSM to it.

Gemma 3 270M and 1B (instruct) were in the plan and did not run: their licence was not accepted on the account that ran the notebook.

Public Kiswahili benchmarks

Belebele (reading, four choices), AfriXNLI (inference, three) and MasakhaNEWS (news topic, seven), scored by suffix likelihood in float32 on a T4, seed 20260914, with a one-sided test against chance.

Every model beats chance on Belebele. On Belebele this model's interval clears the second generation's (29.0, [26.1, 32.0]) and overlaps Qwen2.5's ([28.4, 34.4]), so it is the best score here without being separated from the next. No model in the comparison beats chance on AfriXNLI; the base comes closest (p = 0.055). MasakhaNEWS is news, which is in the pretraining mix, and the test articles were not checked for overlap.

What it gets wrong

State that changes. Told three guests are coming and then, one at a time, that each has dropped out, it answers Wageni watakaokuja ni Mussa na Yusufu. It keeps what was said first. 2 of 40.

Arithmetic across turns, and two-step word problems. It will work any sum you write down, but it does not reliably decide which sum to write. Thirty eggs a tray, three trays and sixteen loose eggs: it adds 30 and 16 and answers 46. A balance carried over several turns and then spent: 1 of 40. Grade-school problems (AfriMGSM): 1 of 100.

Declining too often. It declined 4 of 30 easy questions it should have answered, among them when Tanganyika became independent and which ocean Zanzibar is in. Of the 30 it answered 21 correctly. The fine-tuning taught declines on obscure facts, and some of that reached plain ones.

Facts. It is a 230M model. The base states Kilimanjaro's height and Nyerere's birthday confidently and wrongly. Treat every factual claim that is not in your prompt as unverified.

Instructions that should persist. Told to answer every following question in one word, it often answers the next one in full (Bahari ya Hindi.): 7 of 20.

Scope. Kiswahili only, mostly Tanzanian, and part of the corpus is machine translated. Not English, not code, not fractions or decimals.

Training

Pretraining (KW5-230M):

Fine-tuning (this model): full-parameter, two epochs of 204,483 conversations, 3,195 steps, context 2,048, assistant turns only, from the base and at the base's RoPE (theta 204,522). The mix:

• curated Kiswahili instruction sets, Aya, MURI-IT and TyDiQA train, each read source by source and cut where it taught the wrong thing;
• grounded questions over real passages, including questions over another article's passage that are declined;
• generated arithmetic with its working, long conversations with facts told and never told, changing lists and persistent instructions, every equation checked by a test;
• about 10,700 conversations translated by Gemma 4 26B-A4B, kept only when every turn passed a check, and question-answer pairs it wrote over FineWiki;
• calibrated declines: PolyFact-sw questions the first fine-tune got right at most once in eight samples became declines (capped at 2,000), and those it got right six or more times became answers;
• 10,213 of the first fine-tune's own replies that passed a verifier (right number with every equation true, span present, exact count).

Declines are 10.4% of the mix. Any conversation whose user turn matched an evaluation prompt was dropped (224). General Kiswahili was probed throughout: the held-out loss drifted by +0.054 nats against a tolerance of 0.35.

One thing to know about the scores above. This mix was built after the first fine-tune was scored on the same frozen suite, to cover what it missed: imperative arithmetic phrasings, two-step problems, lists acknowledged without being echoed, declines in TyDiQA's format. The evaluation items themselves were kept out by hash and by test, but the suite's arithmetic is no longer novel in frame, only in wording. A held-out suite written after this release is the next measurement.

Not yet measured

• Safety. There is no dedicated safety tuning beyond declining, and the over-refusal and harmful-request sets (XSTest-sw, MultiJail-sw) have not been run on this release. Do not put it in front of the public without your own safeguards.
• Speed on a CPU. The second generation runs on a laptop CPU at 149M; this one is 1.3x its stored size and has not been timed.

Intended use

Research, and Kiswahili work where the answer is in the prompt and a person checks the output: questions about a supplied passage, working a sum someone wrote down, short conversations that refer back to earlier turns. A base for fine-tuning on your own Kiswahili data.

Not for factual lookup, multi-step word problems, or anything where a confident wrong answer costs something.

Citation

@misc{regnant2026kw5v3,
  title  = {KW5-230M: a Kiswahili language model from the tokenizer up},
  author = {Regnant},
  year   = {2026},
  url    = {https://huggingface.co/regnant-io/kw5-230M-instruct}
}

Apache 2.0.