whisper-small-sinhala-v6-e6-run11-best
Full fine-tune of openai/whisper-small (244M parameters) for Sinhala automatic speech recognition. It reaches 16.38% WER / 4.43% CER on a speaker-disjoint test set of 15,860 utterances.
It is the ASR model behind SinhaSpeech, a Sinhala accessibility platform built as a University of Moratuwa DSE project. The platform covers live lecture captioning, transcript review, spoken quizzes and a hands-free voice-command mode.
Usage
from transformers import pipeline
asr = pipeline(
"automatic-speech-recognition",
model="SinhaSpeech/whisper-small-sinhala-v6-e6-run11-best",
chunk_length_s=30,
)
print(asr("sample.wav", generate_kwargs={"language": "sinhala", "task": "transcribe"})["text"])
- Audio should be 16 kHz mono. The pipeline resamples common formats through ffmpeg.
- Whisper handles 30 s windows. Use
chunk_length_s=30, or window the audio yourself for longer recordings. - For low-latency use, the project serves a CTranslate2 (
faster-whisper) conversion of this model for live captioning.
Training
| Base model | openai/whisper-small |
| Method | Full fine-tune (every weight updated, no LoRA) |
| Data | stratified_v6 of SinhaSpeech/sinhala-asr-data (OpenSLR-52 plus a collected set) |
| Split sizes | 123,205 train / 15,763 validation / 15,860 test, speaker-disjoint |
| Learning rate | 3e-5, linear decay, 500 warmup steps |
| Batch size | 64 effective (64 per device x 1 accumulation step) |
| Epochs / steps | 6 epochs, 11,556 steps |
| Precision / hardware | bf16 on one AMD Instinct MI300X (ROCm) |
| Augmentation | SpecAugment on. Per-clip, each with probability 0.2: Gaussian noise (SNR 20-30 dB), time stretch (0.9-1.1x), pitch shift (+/-2 semitones) |
| Generation | language="sinhala", task="transcribe", generation_max_length=225 |
| Model selection | Best validation WER (epoch 6) |
| Train runtime | about 5.75 h (final train loss 0.0356, average train loss 0.114) |
| Final checkpoint | step 11,556 (this is also the best checkpoint) |
About the data version. stratified_v6 canonicalizes about 2,954 word instances compared with stratified_v5:
- letter-level spellings such as ළ/ල and ණ/න
- ZWJ/rakaransaya conjuncts
- word-level register pairs such as the -නං/-නම් conditional ending
- English typos in code-switched references
The full motivation and method are in the project's error analysis.
Results
| Split | WER | CER |
|---|---|---|
| Validation (epoch 6, 15,763 rows) | 19.61% | 6.17% |
| Test (15,860 rows) | 16.38% | 4.43% |
Test WER is lower than validation WER here. The two splits are different speaker-disjoint partitions, so they are not expected to match exactly.
Training curves
Train loss falls smoothly from 2.59 to 0.036. Eval loss falls from 0.220 to 0.128 by epoch 5, then ticks up slightly to 0.132 at epoch 6 while WER and CER still improve. This is a mild divergence worth watching, not clear overfitting.
| Epoch | Val WER | Val CER | Val loss |
|---|---|---|---|
| 1 | 29.44% | 8.75% | 0.2197 |
| 2 | 25.11% | 7.50% | 0.1771 |
| 3 | 22.60% | 6.74% | 0.1606 |
| 4 | 21.20% | 6.48% | 0.1327 |
| 5 | 20.04% | 6.20% | 0.1284 |
| 6 | 19.61% | 6.17% | 0.1316 |
Comparison with other runs
| Run | Setup | Test WER | Test CER |
|---|---|---|---|
| run1 | v1 data (different test split), full FT, 4 ep | 17.08% | 3.49% |
| run5 | v4 data (speaker-disjoint), 4 ep | 19.06% | 4.90% |
| run6-v5 | v5 data, 5 ep | 17.15% | 4.62% |
run10-v6 (whisper-small-sinhala-v6-e5) |
v6 data, 5 ep | 17.17% | 4.71% |
| run11-v6 (this model) | v6 data, 6 ep | 16.38% | 4.43% |
WER is only comparable between runs that share a test split. run1 used the v1 split. All other runs shown use v4 or later.
The extra sixth epoch is what produced the gain. run10 (same config, 5 epochs) only tied run6-v5, so the v6 data cleanup alone did not move test WER. The gain over both 5-epoch runs is about 0.8 points WER and 0.2-0.3 points CER.
Repository contents
The repo root holds the final model (model.safetensors, tokenizer and config files) and the training-curve images in images/. Intermediate training checkpoints were removed to keep the repo small.
Citation
@misc{jayasinghe2026whispersinhala,
author = {Yohan Jayasinghe},
title = {whisper-small-sinhala-v6-e6-run11-best: Whisper-small fine-tuned for Sinhala ASR},
year = {2026},
url = {https://huggingface.co/SinhaSpeech/whisper-small-sinhala-v6-e6-run11-best}
}
Built on Whisper (Radford et al., 2022). Training data includes OpenSLR-52 (Sinhala).