MORENA 0.5B mini

A 503M-parameter model cut from MORENA 1.5B base in one pruning pass (28 layers to 16, channels and attention heads scored by contribution) and repaired by distilling 10.5B tokens against the teacher's full output distribution. Cost: 212 A100 GPU-hours. Not a chat model.

	MORENA 0.5B mini	reference
African bits per byte, mean of 12	1.520	InkubaLM-0.4B 2.960, gemma-3-1b-pt 2.335, gemma-3-12b-it 2.159
best under 1B parameters on	11 of 12 languages	exception isiNdebele: mzansilm-125m 2.783 against the mini's 2.862
beats the 8B Lugha-Llama-8B on	isiZulu only	1.318 against 1.323
ARC-easy (English)	0.46	teacher (1.5B base) 0.676: about a third of the reasoning was lost
Translation chrF++, en to African / African to en, 3-shot	22.4 / 21.9
decode speed, one A100	2.85 ms/char

On African multiple-choice tasks it is at chance, like its teacher and every other model under 12B measured. It has not been safety-tested or put in front of a person. A GGUF build is in vamboai/morena-0.5b-mini-gguf.

The MORENA family

model	params	African bpb (all 12, lower is better)	role
MORENA 1.5B base	1.485B	1.408	pretrained and mid-trained; fine-tuning starting point
MORENA 1.5B instruct	1.485B	1.441	chat, translation, tool calling; the model described in the paper
MORENA 0.5B mini	503M	1.520	pruned and distilled from the 1.5B base
MORENA 0.5B mini instruct	503M	1.540	chat fine-tune of the mini
MORENA 0.2B nano	209M	1.583	cheap trunk for ASR rescoring, keyboards, normalisation

26 models in total were measured on African bits per byte, 21 of them outside models from 125M to 12B parameters. MORENA 1.5B base is lowest of all 26 at 1.408; the 8B African specialist Lugha-Llama-8B is second at 1.423, ahead of the other four MORENA sizes on the twelve-language mean. Every general-purpose model measured, from 1B to 12B parameters, trails well behind: the best of them, gemma-3-12b-it, reaches only 2.159. Twelve languages: Shona, Swahili, Hausa, Yoruba, Igbo, isiZulu, isiXhosa, Kinyarwanda, Setswana, Afrikaans, isiNdebele and Nigerian Pidgin, plus English, French and code. isiNdebele (ISO code nbl) has no FLORES+ or MasakhaNEWS coverage and is evaluated on NCHLT transcripts only.

Tokenizer: 65,536-entry byte-fallback BPE trained on the target mix. African text costs 0.249 tokens per byte against 0.234 for English, about 6% more per byte than English in MORENA's vocabulary, but that same African text needs 1.39x fewer tokens than under Gemma 3's vocabulary and 1.53x fewer than under Llama 3.2's.

Translation is measured for seventeen models in total, thirteen of them outside models.

Author and citation

Isheanesu Misi, Vambo AI. Trained on CINECA Leonardo, with support from the AI Hub for Sustainable Development.

@techreport{misi2026morena,
  title  = {MORENA: An African Foundation Model},
  author = {Misi, Isheanesu},
  institution = {Vambo AI},
  year   = {2026},
  month  = {September},
  note   = {Released 18 September 2026}
}

Licence

Apache 2.0. Commercial use is permitted, including fine-tuning and redistributing derivatives, under the terms of the licence.

The training corpus is not released with these weights.