Whisper-small Luganda · Agricultural ASR (Stage 2 + CV25 Rehearsal)
E3b in the experiment matrix of the MSc dissertation "Building an ASR Model for Luganda Agricultural Information Access" (Makerere University) — a follow-up to E3, which showed that naive Stage-2 fine-tuning on Yogera alone caused catastrophic forgetting of general Luganda (T1 WER +7.87pp) for essentially no gain on the agricultural target (T2 WER -0.18pp, within noise). E3b adds two targeted changes aimed directly at those findings — CV25 rehearsal mixed into training, and a dual-domain early-stopping signal — see Training details below.
openai/whisper-small
│ Stage 1 — general Luganda (Common Voice 25 train, 15k steps, LR 1e-5)
▼
Gemmar/luganda-agr-asr-whisper-small-stage1
│ Stage 2 — agricultural Luganda + CV25 rehearsal (LR 3e-06, early stopping)
▼
Gemmar/luganda-agr-asr-whisper-small-stage2-rehearsal ← this model
Results
| Test set | WER | CER | 95% CI (WER) |
|---|---|---|---|
| T2 — Yogera Agricultural (n=11,679) | 34.92% | 13.09% | [34.47, 35.42] |
| T1 — Common Voice 25 general (n=11,823) | 52.7% | 50.56% | [42.74, 62.55] |
Stage-2 effect vs the Stage-1 model (Gemmar/luganda-agr-asr-whisper-small-stage1): T2 WER 34.94% → 34.92% (Δ -0.02 pp) · T1 WER 46.50% → 52.7% (Δ 6.20 pp, catastrophic-forgetting check).
Bootstrap CIs: 1,000 resamples with replacement, seed 42. Text normalisation: lowercase, punctuation stripped (apostrophes kept), whitespace collapsed — identical across all nine experimental conditions (E1–E9).
Usage
from transformers import pipeline
asr = pipeline("automatic-speech-recognition", model="Gemmar/luganda-agr-asr-whisper-small-stage2-rehearsal")
# Force the Luganda decoding prefix (custom token added during Stage 1):
tok = asr.tokenizer
asr.model.generation_config.forced_decoder_ids = [
[1, tok.convert_tokens_to_ids("<|luganda|>")],
[2, tok.convert_tokens_to_ids("<|transcribe|>")],
[3, tok.convert_tokens_to_ids("<|notimestamps|>")],
]
asr.model.generation_config.no_repeat_ngram_size = 3
print(asr("farmer_question.wav")["text"])
Training details
- Stage 2 data: Yogera Agricultural Speech Corpus train split, mixed with a 15% slice of CV25 general-Luganda train (shuffled together, not phased) so the model keeps seeing general Luganda throughout Stage 2 instead of training on the agricultural domain exclusively. The Yogera and CV25 test splits are both fully held out.
- Recipe: LR 3e-06 (cosine, 300 warmup), effective batch 32, bf16, dropout 0.1, label smoothing 0.1, SpecAugment, weight decay 0.01, early stopping (patience 5) on a dev set that mixes a 3,000-sample Yogera-dev subset with a 20% slice of CV25-dev, seed 42 — so checkpoint selection can no longer favour an agricultural-only gain that comes at the cost of general-domain forgetting.
- Luganda conditioning: custom
<|luganda|>token (added in Stage 1, initialised as the mean of Whisper's 99 language embeddings) prepended to all training labels and forced at inference. - Full training telemetry: W&B run
Intended use & limitations
Built for agricultural information access in Uganda — transcribing farmers' spoken Luganda queries for advisory/retrieval systems. Trained on clean crowdsourced prompts; not validated on noisy field recordings, code-switched speech, or other Ugandan languages. Research model — verify before production use.
Citation
Gemmar, F. (2026). Building an ASR Model for Luganda Agricultural Information
Access. MSc dissertation, Makerere University.