SunflowerASR: speech recognition for 51 African languages

SunflowerASR is an adaptation of Whisper large-v3 that supports Automatic Speech Recognition (ASR) for 51 languages widely spoken in Africa: English and French with African accents, Swahili, Afrikaans, Tswana, Kinyarwanda, Nigerian Pidgin, Luganda, Acholi, Lugbara, Ateso, Runyankole, Rutooro, Lumasaba, Lusoga, Rukiga, Akan, Amharic, Bambara, Bemba, Berber, Chichewa, Dagaare, Dagbani, Ewe, Fulani, Hausa, Igbo, Ikposo, Kabyle, Kalenjin, Kanuri, Kikuyu, Kwamba, Lendu, Lingala, Luhya, Luo, Malagasy, Ndebele, Oromo, Rukonjo, Ruruuli, Shona, Somali, Sotho, Thur, Wolof, Xhosa, Yoruba, and Zulu.

It was trained by Sunbird AI in two stages: full-parameter supervised fine-tuning on 7,412 hours of African speech, followed by a reinforcement learning pass with Group Relative Policy Optimisation (GRPO) that optimises character error rate directly. It is released as part of Sunflower V2, and the method is described in the accompanying paper (to be released later).

Across three benchmarks it is the most accurate published system we have measured: it has the lowest macro-averaged Word Error Rate (WER) of any model, open or closed, on the Sunbird Speech Benchmark (51 languages), AfriVox-v2 and SimbaBench, including against Meta's Omnilingual ASR, Gemini, GPT-4o-transcribe, Sahara v2 and Simba. Per-language rankings vary, and quality across the 51 languages is very uneven — check the per-language number for your language before relying on it.

Intended use

The model is intended for transcribing speech in the 51 languages listed above, including for applications such as media monitoring, call-centre and citizen-feedback analysis, agricultural and health advisory services, voice interfaces, and as a component in speech translation or speech-to-speech pipelines. It is also intended as a starting point for further fine-tuning on a specific language, domain or accent.

The model transcribes one language per clip: the language token is supplied at decode time and is not detected automatically. It does not translate, does not produce timestamps or speaker labels in this configuration, and is not a speaker-identification, language-identification or voice-biometric system. It should not be used as the sole basis for consequential decisions about individuals (for example in legal, clinical, immigration or employment settings) without human review of the transcript, particularly for the languages in the weaker half of the results tables.

Usage

You can use this Colab Notebook to try out the model. It lets you transcribe an audio sample from a Hugging Face dataset, an uploaded audio file, or your own recording from your computer's microphone.

The model is used in much the same way as the base Whisper model. For better accuracy, specify the language during generation.

See the example below, which transcribes an audio sample from a Hugging Face dataset:

import transformers
import datasets
import torch

SAMPLE_RATE = 16000

LANGUAGE_TOKENS_WHISPER = {
    # Existing languges codes from Whisper
    "eng": 50259, "fra": 50265, "swa": 50318, "sna": 50324, "yor": 50325, "som": 50326,
    "afr": 50327, "amh": 50334, "mlg": 50349, "lin": 50353, "hau": 50354,
    # Overwrite unused language tokens
    "ach": 50357, "aka": 50356, "bam": 50355, "bem": 50352, "ber": 50351,
    "cgg": 50350, "dag": 50348, "dga": 50347, "ewe": 50346, "ful": 50345,
    "ibo": 50344, "kab": 50343, "kau": 50342, "kik": 50341, "kin": 50340,
    "kln": 50339, "koo": 50338, "kpo": 50337, "led": 50336, "lgg": 50335,
    "lth": 50333, "lug": 50332, "luo": 50331, "luy": 50330, "myx": 50329,
    "nbl": 50328, "nya": 50323, "nyn": 50322, "orm": 50321, "pcm": 50320,
    "ruc": 50319, "rwm": 50317, "sot": 50316, "teo": 50315, "tsn": 50314,
    "ttj": 50313, "wol": 50312, "xho": 50311, "xog": 50310, "zul": 50309
}

LANGUAGE_NAMES = {
    'Acholi': 'ach', 'Afrikaans': 'afr', 'Akan': 'aka', 'Amharic': 'amh', 'Ateso': 'teo',
    'Bambara': 'bam', 'Bemba': 'bem', 'Berber': 'ber', 'Chichewa': 'nya', 'Dagaare': 'dga',
    'Dagbani': 'dag', 'English': 'eng', 'Ewe': 'ewe', 'French': 'fra', 'Fulani': 'ful',
    'Hausa': 'hau', 'Igbo': 'ibo', 'Ikposo': 'kpo', 'Kabyle': 'kab', 'Kalenjin': 'kln',
    'Kanuri': 'kau', 'Kikuyu': 'kik', 'Kinyarwanda': 'kin', 'Kwamba': 'rwm', 'Lendu': 'led',
    'Lingala': 'lin', 'Luganda': 'lug', 'Lugbara': 'lgg', 'Luhya': 'luy', 'Lumasaba': 'myx',
    'Luo': 'luo', 'Lusoga': 'xog', 'Malagasy': 'mlg', 'Ndebele': 'nbl', 'Nigerian Pidgin': 'pcm',
    'Oromo': 'orm', 'Rukiga': 'cgg', 'Rukonjo': 'koo', 'Runyankole': 'nyn', 'Ruruuli': 'ruc',
    'Rutooro': 'ttj', 'Shona': 'sna', 'Somali': 'som', 'Sotho': 'sot', 'Swahili': 'swa',
    'Thur': 'lth', 'Tswana': 'tsn', 'Wolof': 'wol', 'Xhosa': 'xho', 'Yoruba': 'yor', 'Zulu': 'zul'
}

model_id = "Sunbird/SunflowerASR-51-african-languages"
model = transformers.WhisperForConditionalGeneration.from_pretrained(model_id)
processor = transformers.WhisperProcessor.from_pretrained(model_id)

def transcribe_by_whisper(audio_array, language):
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    input_features = processor(
        audio_array, sampling_rate=SAMPLE_RATE, do_normalize=True, return_tensors="pt"
    ).input_features.to(device)

    lang_tok = LANGUAGE_TOKENS_WHISPER[LANGUAGE_NAMES[language]]
    transcribe_tok = processor.tokenizer.convert_tokens_to_ids("<|transcribe|>")
    notimestamps_tok = processor.tokenizer.convert_tokens_to_ids("<|notimestamps|>")
    forced_decoder_ids = [
        (1, lang_tok),
        (2, transcribe_tok),
        (3, notimestamps_tok),
    ]

    predicted_ids = model.to(device).generate(
        input_features,
        forced_decoder_ids=forced_decoder_ids,
        num_beams=1,
        do_sample=False,
    )
    transcription = processor.decode(
        predicted_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)

    print(transcription[0])

# Get some test audio
import huggingface_hub
huggingface_hub.login()

ds = datasets.load_dataset('Sunbird/salt', 'multispeaker-lug', split='test')
audio_column = 'audio'
ds = ds.cast_column(audio_column, datasets.Audio(sampling_rate=SAMPLE_RATE))
audio_array = ds[0][audio_column]['array']

# Specify a language
lang = 'Luganda'

transcribe_by_whisper(audio_array, lang)
# Ekikoola kya kasooli kya kyenvu wabula langi yakyo etera okuba eya kitaka wansi.

Two decoding notes. The language token must be forced from the table above rather than passed as a Whisper language name: the 40 languages Whisper did not originally cover are mapped onto unused token ids, so language="lug" alone will not select Luganda. And if you set max_new_tokens, keep it at 444 (Whisper's architectural ceiling of 448 decoder positions, minus the 4-token forced prefix) — Whisper's tokeniser has no Ge'ez coverage and falls back to bytes, so Amharic costs about 2.4 tokens per character against 0.4 for a median language here, and a shorter budget truncates its transcripts.

Performance metrics

We evaluate SunflowerASR on three benchmarks: our own Sunbird Speech Benchmark, which is the only one that covers the model's full language range, and the two independent benchmarks with published baselines, AfriVox-v2 and SimbaBench. WER and CER are computed with greedy decoding, the language token forced at the decoder, and Whisper's BasicTextNormalizer applied to both hypothesis and reference. Lower is better throughout.

Sunbird Speech Benchmark

The Sunbird Speech Benchmark is assembled from the held-out test splits of the 110 dataset subsets of our training corpus, capped at 200 clips each: 20,213 clips over 51 languages drawn from 28 source corpora. It is the only one of the three benchmarks that covers every language the model supports. Note that because it is drawn from the same collections as our training data, it is in-domain for SunflowerASR and out-of-domain for the baselines; read it as a measure of coverage across 51 languages, with AfriVox-v2 and SimbaBench below providing the arm's-length comparisons.

Macro-averages over languages, comparing against Omnilingual ASR from Meta, Simba-S from UBC-NLP, Gemini 3.5 Flash from Google and GPT-4o-transcribe from OpenAI. Systems cover different subsets of the 51 languages, so the last column repeats SunflowerASR's average over exactly the languages each baseline scores:

SunflowerASR has the lowest WER of any system compared on 30 of the 51 languages, and the lowest CER on 34.

Per-language results, sorted by WER, alongside the hours of supervised training audio that language contributed to the corpus:

AfriVox-v2

AfriVox-v2 is an in-the-wild benchmark of 251 hours aggregating the Africa Next Voices, Waxal and AfriVox v1 corpora over a 10-domain taxonomy spanning health, finance, government and agriculture. It is largely spontaneous rather than read speech. We score 111,051 clips in the 14 languages that have metrics reported in the AfriVox-v2 paper; baseline figures are taken from that paper, which reports WER only. Simba-S is our own evaluation, and does not cover three of the languages, so its average is over the other eleven. Best per row in bold.

SimbaBench

SimbaBench aggregates read and conversational speech from Common Voice, BembaSpeech, Lwazi, NCHLT and regional collections. The table covers the 19 languages in common with SunflowerASR; baseline figures come from the public leaderboard. Best per row in bold.

Training Datasets

The model was fine-tuned on a multilingual corpus covering 51 African languages, assembled from a range of publicly available and community-collected speech datasets. The table below lists each source dataset and the languages it contributes to training (ISO 639-3 code in parentheses). A language may appear in more than one source.

Corpus statistics

Per-language hours are listed alongside the benchmark results in the Sunbird Speech Benchmark table above. Clips are filtered and then packed: short clips from the same subset are concatenated to fill the 30 s input window, which keeps every pack monolingual and single-source while cutting padding waste from 76.7% to 19.4%. Of the 113 training subsets, the 110 with held-out test splits make up the Sunbird Speech Benchmark.

Sources

Notes

• African Next Voices is drawn from two collection hubs: Kenya (Kikuyu, Kalenjin, Luo, Somali) and Southern Africa (Ndebele, Sotho, Tswana, Xhosa, Zulu).

Training procedure

SunflowerASR is trained from Whisper large-v3 in two stages. In the first, all 1.55 B parameters are fine-tuned for 5 epochs over the packed corpus described above, with a per-device batch of 64 packed samples and 4 gradient accumulation steps — an effective batch of 256 packed samples, or roughly 1.7 hours of audio per optimiser step, giving about 4,300 optimiser steps per epoch — at a peak learning rate of 2×10⁻⁵ with 100 warmup steps and cosine decay, label smoothing of 0.1, and augmentation applied on the fly: additive noise drawn from a corpus of Ugandan ambient recordings (Sunbird/urban-noise-uganda-61k) plus speed and bandwidth perturbation. The second stage is a reinforcement learning pass with Group Relative Policy Optimisation (GRPO), which optimises the sequence-level metric directly rather than the per-token likelihood of a single reference: for each clip the policy samples 10 complete transcripts, each is scored with the reward -min(CER, 1.0) against the reference, and the group's own mean and standard deviation turn those scores into advantages, so probability mass moves towards the better rollouts under a KL penalty of β = 0.01 against the initial policy. It runs for one epoch over 15,285 clips, selected as the 200 clips per dataset subset with the largest headroom — the gap between the median and the best CER over 10 sampled rollouts, which identifies clips where a better transcript is already inside the model's sampling distribution and only needs to be made more probable — with 64 distinct clips (640 rollouts) per update at a peak learning rate of 10⁻⁵, the encoder frozen so that only the decoder's 906 M parameters are trained. This stage leaves clean-speech accuracy roughly unchanged but improves noisy and spontaneous audio: it largely suppresses the repetition loops Whisper falls into under acoustic stress, makes ordinary substitutions closer to the reference spelling, and helps most for the languages with the least training data.

Training configuration

Limitations

• Quality is very uneven across the 51 languages. On the Sunbird Speech Benchmark, WER ranges from 0.08 to 0.67. Check the per-language figures before deploying for any one language, and treat the tail of the table as usable for triage or search rather than for verbatim transcription.
• The language must be specified at decode time. The model does not perform language identification, and giving it the wrong language token degrades output substantially. Language tokens for the 40 languages Whisper did not originally support are overwritten unused tokens, so the mapping in the Usage section must be used rather than Whisper's own language names.
• Audio longer than 30 seconds must be segmented before transcription, as with any Whisper model; longer clips are truncated by the encoder.
• Orthographic convention rather than intelligibility. Both training stages score hypotheses against a single reference transcript, so the model is optimised towards the transcription conventions of these corpora. Languages without a settled orthography inherit that ambiguity, and a transcript may be penalised — or produced — in a spelling a given community would not use.
• Domain and style. Much of the training data for the lower-resource languages is read speech recorded in quiet conditions. Accuracy is lower on spontaneous, overlapping, noisy or heavily code-switched speech, though this is the setting the GRPO stage improves most.
• Repetition loops are reduced, not eliminated. Under heavy acoustic stress the model can still emit a repeated run or a hypothesis much longer than the utterance.
• Benchmark caveat. The Sunbird Speech Benchmark is drawn from the same collections as the training corpus, so it is in-domain for this model and out-of-domain for the baselines it is compared against. AfriVox-v2 and SimbaBench are the arm's-length comparisons.
• Not for consequential decisions without review. See Intended use.

Citation

If you use this model, please cite the accompanying paper:

@misc{sunflowerasr2026,
  title        = {Group Relative Policy Optimisation Improves Multilingual Speech
                  Recognition in Low-Resource Languages},
  author       = {Hu, Tim Wenjie and Ouma, Evelyn Nafula and Akera, Benjamin and
                  Quinn, John A.},
  year         = {2026},
  institution  = {Sunbird AI},
  howpublished = {\url{https://huggingface.co/Sunbird/SunflowerASR-51-african-languages}}
}

And, for the evaluation set:

@misc{sunbird_speech_benchmark_2026,
  title        = {Sunbird Speech Benchmark: a multilingual ASR test set for 51
                  African languages},
  author       = {Sunbird AI},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/datasets/Sunbird/speech-benchmark}}
}

License and acknowledgements

Released under the Apache 2.0 licence, following the licence of the Whisper large-v3 base model. The training corpus is assembled from the publicly available and community-collected datasets credited in Training Datasets; please also respect the licence and citation requirements of those sources. We thank the many organisations and community contributors who collected and released this speech data, without which a model covering these languages would not be possible.