FLEURS-Only Whisper Small No-Language Checkpoint
Summary
This repository contains a Whisper checkpoint for Chichewa/Nyanja
automatic speech recognition, fine-tuned from openai/whisper-small.
- Experiment type:
fleurs-only - Base model:
openai/whisper-small - Training condition:
no_language - Release artifact: full fine-tuned checkpoint selected from the best training checkpoint
Intended use
This checkpoint is intended for research and evaluation on Chichewa/Nyanja ASR. It is not a production-ready speech system and should be validated carefully before downstream use.
How to use
This repository contains a full fine-tuned checkpoint. It can be loaded directly with Transformers.
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
model_id = "ai4good-labyrinth/fleurs-only-whisper-small-no-language"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(model_id)
Training data
- Training source:
TBD - Evaluation source during training:
TBD - Train examples before duration filtering:
TBD - Train examples after duration filtering:
TBD - Dev examples before duration filtering:
TBD - Dev examples after duration filtering:
TBD - Duration filter used during training:
min_duration_seconds=TBD,max_duration_seconds=TBD
Training procedure
- Fine-tuning script:
experiments/whisper_finetune/finetune_whisper.py - Base model:
openai/whisper-small - Task:
transcribe - Language hint during training/evaluation: none, corresponding to
--language autoin standalone evaluation - Mixed precision:
TBD - Gradient checkpointing:
TBD - Selected checkpoint step:
600 - Selected checkpoint epoch:
1.81
Training-time dev selection
The best checkpoint was selected using trainer-side dev evaluation on the duration-filtered FLEURS dev split.
- Dev WER:
0.5521 - Dev CER:
0.1537 - Dev loss:
0.6976
These values come from the training pipeline and may differ slightly from standalone post-hoc evaluation because the decoding path is not perfectly identical.
Evaluation protocol
Standalone evaluation is recommended for the final release. Filtered and unfiltered results should be reported separately.
- Filtered evaluation:
min_duration_seconds=0,max_duration_seconds=30 - Unfiltered evaluation: no duration constraint
- Decoding task:
transcribe - Language hint:
auto
Evaluation summary
| Dataset | Split | Setting | Num examples | WER | CER | Notes |
|---|---|---|---|---|---|---|
| FLEURS | dev | filtered | 305 | 0.6263 | 0.1929 | Filtered to 30 seconds |
| FLEURS | dev | unfiltered | 311 | 0.6198 | 0.1918 | No duration filter |
| FLEURS | test | filtered | 745 | 0.5452 | 0.1597 | Filtered to 30 seconds |
| FLEURS | test | unfiltered | TBD | TBD | TBD | Standalone eval pending |
| Zambezi | dev | filtered | 613 | 0.6593 | 0.1883 | Filtered to 30 seconds |
| Zambezi | dev | unfiltered | 622 | 0.6567 | 0.1868 | No duration filter |
| Zambezi | test | filtered | 427 | 0.6805 | 0.1801 | Filtered to 30 seconds |
| Zambezi | test | unfiltered | 428 | 0.6776 | 0.1790 | No duration filter |
Files in this repository
- Model weights and config: repository root
- Processor/tokenizer files: repository root
- Evaluation JSON files:
eval/...
Known limitations
- Whisper does not provide an official Nyanja/Chichewa language token.
- Users must also comply with the upstream dataset licenses and any upstream model license obligations.
- Standalone evaluation and trainer-side evaluation can differ slightly even on the same split and duration filter.
- Cross-dataset results should be interpreted carefully because transcription conventions may differ across corpora.
Citation
If you use this checkpoint, please cite:
- the Whisper paper
- the FLEURS dataset
- this repository
@misc{fleurs_only_whisper_small_no_language_2026,
title = {FLEURS-Only Whisper Small No-Language Checkpoint},
author = {AI4Good Labyrinth Team},
year = {2026},
howpublished = {\url{https://huggingface.co/ai4good-labyrinth/fleurs-only-whisper-small-no-language}},
note = {Whisper fine-tuning for Chichewa/Nyanja ASR}
}