openai/whisper-small fine-tuned on Common Voice 26.0 — Maltese
openai/whisper-small fine-tuned for Maltese (mt) automatic speech recognition
on Mozilla Common Voice Scripted Speech 26.0, trained on a laptop
(Apple M3 Pro, PyTorch MPS backend, fp32).
Results
| Model | Raw WER | Normalized WER | Test clips |
|---|---|---|---|
| whisper-base fine-tuned | 105.7% | 94.1% | 300 |
| whisper-base zero-shot | 107.6% | 134.7% | 300 |
| whisper-small fine-tuned | 46.7% | 38.9% | 300 |
| whisper-small zero-shot | 110.1% | 100.6% | 300 |
Zero-shot Whisper barely functions on Maltese (≈1 h of it in the
original training mix); the WER delta above is what a few hours of
community-donated Common Voice speech and a laptop fine-tune buy. WER
computed on the official Common Voice test split (speaker-disjoint from
training data — note the training split draws on only 6
speakers, which limits generalization; see Limitations). "Normalized"
applies BasicTextNormalizer (lowercase, strip punctuation) to both
reference and hypothesis.
Data provenance
- Source: Mozilla Common Voice Scripted Speech 26.0,
Maltese (mt), obtained via the
Mozilla Data Collective
(since October 2025 the exclusive distribution channel for Common Voice
datasets — the former
mozilla-foundation/common_voice_*datasets on this Hub are empty shells). - License: the underlying speech data is CC0.
- Used here: 3000 training clips (~3.77 h, 6 speakers), 200 dev clips, 300 test clips, official splits, clips 1–15 s.
- The prepared audio dataset is deliberately not re-uploaded to this Hub; get the data from the Mozilla Data Collective.
Training
Standard Hugging Face Seq2SeqTrainer recipe
(fine-tune-whisper):
batch 4 × grad-accum 4, lr 1e-05, warmup 50,
1000 steps, fp32 (MPS), gradient checkpointing, greedy decoding,
best-checkpoint-by-WER selection.
Limitations
- Single low-resource language, ~3.77 h of scripted (read) speech from only 6 speakers (the corpus's many one-clip contributors are concentrated in the held-out splits) — expect degradation on spontaneous/conversational audio and unseen voices.
- Small test set (300 clips); WER has meaningful variance.
- No punctuation/casing guarantees; laptop-scale training run, not a production model.
Usage
from transformers import pipeline
asr = pipeline("automatic-speech-recognition", model="titaniumbones/whisper-small-mt-commonvoice")
print(asr("clip.mp3", generate_kwargs={"language": "maltese"}))