FLEURS-Trigram Whisper Small No-Language Checkpoint
Summary
This repository contains a Whisper checkpoint for Chichewa/Nyanja
automatic speech recognition, fine-tuned from openai/whisper-small.
- Experiment type:
fleurs-trigram - Base model:
openai/whisper-small - Training condition:
no_language - Release artifact: full fine-tuned checkpoint selected from the best training checkpoint
Intended use
This checkpoint is intended for research and evaluation on Chichewa/Nyanja ASR. It is not a production-ready speech system and should be validated carefully before downstream use.
How to use
This repository contains a full fine-tuned checkpoint. It can be loaded directly with Transformers.
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
model_id = "ai4good-labyrinth/fleurs-trigram-hours1p00-whisper-small-no-language"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id)
Training data
- Training source:
FLEURS train + Chichewa Trigrams train - Evaluation source during training:
FLEURS dev - Train examples before duration filtering:
28629 - Train examples after duration filtering:
28588 - Dev examples before duration filtering:
311 - Dev examples after duration filtering:
305 - Duration filter used during training:
min_duration_seconds=0.0,max_duration_seconds=30.0
Training procedure
- Fine-tuning script:
experiments/whisper_finetune/finetune_whisper.py - Base model:
openai/whisper-small - Task:
transcribe - Language hint during training/evaluation: none, corresponding to
--language autoin standalone evaluation - Mixed precision:
no - Gradient checkpointing:
False - Selected checkpoint step:
7500 - Selected checkpoint epoch:
2.10
Training-time dev selection
The best checkpoint was selected using trainer-side dev evaluation on the duration-filtered FLEURS dev split.
- Dev WER:
0.4776 - Dev CER:
0.1261 - Dev loss:
0.6753
These values come from the training pipeline and may differ slightly from standalone post-hoc evaluation because the decoding path is not perfectly identical.
Evaluation protocol
Standalone evaluation is recommended for the final release. Filtered and unfiltered results should be reported separately.
- Filtered evaluation:
min_duration_seconds=0,max_duration_seconds=30 - Unfiltered evaluation: no duration constraint
- Decoding task:
transcribe - Language hint:
auto
Evaluation summary
| Dataset | Split | Setting | Num examples | WER | CER | Notes |
|---|---|---|---|---|---|---|
| FLEURS | dev | filtered | 305 | 0.5337 | 0.1607 | Filtered to 30 seconds |
| FLEURS | dev | unfiltered | 311 | 0.5269 | 0.1580 | No duration filter |
| FLEURS | test | filtered | 745 | 0.4822 | 0.1396 | Filtered to 30 seconds |
| FLEURS | test | unfiltered | 761 | 0.4931 | 0.1446 | No duration filter |
| Zambezi | dev | filtered | 613 | 0.6385 | 0.1932 | Filtered to 30 seconds |
| Zambezi | dev | unfiltered | 622 | 0.6350 | 0.1913 | No duration filter |
| Zambezi | test | filtered | 427 | 0.6741 | 0.1945 | Filtered to 30 seconds |
| Zambezi | test | unfiltered | 428 | 0.6703 | 0.1932 | No duration filter |
Files in this repository
- Model weights and config: repository root
- Processor/tokenizer files: repository root
- Evaluation JSON files:
eval/...
Known limitations
- Whisper does not provide an official Nyanja/Chichewa language token.
- Users must also comply with the upstream dataset licenses and any upstream model license obligations.
- Standalone evaluation and trainer-side evaluation can differ slightly even on the same split and duration filter.
- Cross-dataset results should be interpreted carefully because transcription conventions may differ across corpora.
Citation
If you use this checkpoint, please cite:
- the Whisper paper
- the FLEURS dataset
- this repository
@misc{fleurs_trigram_hours1p00_whisper_small_no_language_2026,
title = {FLEURS-Trigram Whisper Small No-Language Checkpoint},
author = {AI4Good Labyrinth Team},
year = {2026},
howpublished = {\url{https://huggingface.co/ai4good-labyrinth/fleurs-trigram-hours1p00-whisper-small-no-language}},
note = {Whisper fine-tuning for Chichewa/Nyanja ASR}
}