Whisper Large V3 — Multilingual Luganda + Ugandan English NSS (ml-v1)
Stage 2 multilingual fine-tune on the CDLI Ugandan Nonstandard Speech datasets. Built on top of lg_waxal_tune_whisper_large_7 (Stage 1). Trained jointly on Luganda and Ugandan English NSS data. Evaluated on Luganda NSS test set.
Training Details
- Datasets: cdli/ugandan_luganda_nonstandard_speech_v1.0 + cdli/ugandan_english_nonstandard_speech_v1.0
- Stage 1 checkpoint: lg_waxal_tune_whisper_large_7/best_model
- Encoder frozen: No
- Decoder frozen: No
- Projection frozen: No
- SpecAugment: On
- Data augmentation: Yes (Gaussian noise, gain perturbation, speed perturbation)
- Learning rate: 5e-5 (polynomial decay)
- Max steps: 1000
- Best checkpoint: Step 850
Results on Luganda NSS Test Set (checkpoint-850)
| Severity | Baseline WER | Fine-tuned WER |
|---|---|---|
| Mild | 0.89 | 0.50 |
| Moderate | 0.96 | 0.53 |
| Severe | 0.97 | 0.60 |
| Etiology | Baseline WER | Fine-tuned WER |
|---|---|---|
| Acquired hearing impairment | 0.98 | 0.67 |
| Cerebral Palsy | 0.95 | 0.43 |
| Cerebral palsy - cerebral malaria | 0.93 | 0.55 |
| Developmental | 0.97 | 0.54 |
| Idiopathic | 0.80 | 0.45 |
| Structural (tongue tie) | 0.96 | 0.65 |
- Overall WER (normalized): 0.745
- Overall CER (normalized): 0.290
- Avg WER (normalized): 0.529
- Avg CER (normalized): 0.182
- Meaning preservation: 85.1%