MORENA 1.5B instruct

MORENA, Sesotho and Setswana for a king, a lord or a chief, is a 1.5B-parameter decoder trained from scratch for twelve Latin-script African languages plus English, French and code. This is the instruction-tuned model described in the paper MORENA: An African Foundation Model.

Headline numbers

MORENA 1.5B instruct's English-into-African chrF++ of 45.8 is eight points above MADLAD-400-3B (37.8).

Safety by category (share of all attempts handled well, 24 prompts per category per language): self-harm 92.5, child safety 91.2, drugs 86.2, violence 88.9, election 89.2, privacy 93.8, fraud 94.3, weapons 93.3, hate 95.4, medication 95.4. Weakest languages: Igbo 75%, Yoruba 82%, Setswana 83%, isiXhosa 87%. Every number above is a model (google/gemma-3-12b-it) judging a model; no native speaker has yet rated an answer.

What it is not good at

Retrieval-augmented QA is at chance in African languages even though the model demonstrably reads the passage (grounding 0.73). Grounded generation is fully faithful to given facts in 23% of attempts, down from 31% in the previous version. The model fails to answer 41% of ordinary benign requests well, most of them by refusing; an earlier version once refused to recommend a dry cleaner, citing "illegal substances or services". That specific failure is fixed, but the broader over-refusal problem is not. Tool calling is measured with the tool marker prefilled; left to decide for itself the model almost never calls one. Multiple-choice comprehension in African languages is at chance for this model and for every model under 12B measured.

The checkpoint was chosen against a rule fixed before the final experiments: every harm category within 3.5 points of the previous version, refusals on at least 91% of all attempts, degeneracy at most 1%, no capability lost. It misses that rule on privacy by 0.1 points, on drugs by 1.0 and on violence by 2.6, and on degeneracy, which is 1.3%.

Chat format

Single reserved tokens mark turns: <reserved_0> opens a user turn and <reserved_1> an assistant turn (token ids 3 and 4). load_example.py in this repo shows a full prompt. Do not use <|user|>-style strings; they are not in the vocabulary and produce degenerate output.

Files

model.safetensors (bf16), config.json, tokenizer.json, modeling_morena.py (reference implementation, plain PyTorch, no transformers dependency), load_example.py, SHA256SUMS. A GGUF build for llama.cpp is in vamboai/morena-1.5b-instruct-gguf.

Training

251.7B tokens of pretraining, 63B tokens of mid-training (315B tokens seen in total; see MORENA 1.5B base), then 4,500 steps of supervised fine-tuning with loss masking and a 500-step safety anneal. Training mixture moved in three regimes as machine-translated languages landed: African text was 24.8% of tokens seen (14.8% machine-translated) for steps 1 to 25,304, 39.1% (31.0% machine-translated) for steps 25,305 to 60,000, and 50.2% (41.6% machine-translated) during mid-training. Nine languages were machine-translated from English documents: this is 24% of pretraining tokens seen and 28% including mid-training (57.1B tokens on disk, 21% of the 271B on disk).

Release lineage: 12,834 A100 GPU-hours, an estimated twenty-five to forty thousand dollars at $2 to $3 per A100-hour; the research programme that produced it, about 22,450 GPU-hours (about 22% of a 100,000 GPU-hour allocation). Architecture: 28 layers x 2048, GQA 16/4, SwiGLU 6144, RoPE theta 500,000, 4,096 context, tied embeddings. Optimiser: Muon for non-embedding weights, AdamW for the rest, warmup-stable-decay schedule.

The MORENA family

26 models in total were measured on African bits per byte, 21 of them outside models from 125M to 12B parameters. MORENA 1.5B base is lowest of all 26 at 1.408; the 8B African specialist Lugha-Llama-8B is second at 1.423, ahead of the other four MORENA sizes on the twelve-language mean. Every general-purpose model measured, from 1B to 12B parameters, trails well behind: the best of them, gemma-3-12b-it, reaches only 2.159. Twelve languages: Shona, Swahili, Hausa, Yoruba, Igbo, isiZulu, isiXhosa, Kinyarwanda, Setswana, Afrikaans, isiNdebele and Nigerian Pidgin, plus English, French and code. isiNdebele (ISO code nbl) has no FLORES+ or MasakhaNEWS coverage and is evaluated on NCHLT transcripts only.

Tokenizer: 65,536-entry byte-fallback BPE trained on the target mix. African text costs 0.249 tokens per byte against 0.234 for English, about 6% more per byte than English in MORENA's vocabulary, but that same African text needs 1.39x fewer tokens than under Gemma 3's vocabulary and 1.53x fewer than under Llama 3.2's.

Translation is measured for seventeen models in total, thirteen of them outside models.

Author and citation

Isheanesu Misi, Vambo AI. Trained on CINECA Leonardo, with support from the AI Hub for Sustainable Development.

@techreport{misi2026morena,
  title  = {MORENA: An African Foundation Model},
  author = {Misi, Isheanesu},
  institution = {Vambo AI},
  year   = {2026},
  month  = {September},
  note   = {Released 18 September 2026}
}

Licence

Apache 2.0. Commercial use is permitted, including fine-tuning and redistributing derivatives, under the terms of the licence.

The training corpus is not released with these weights.