Whisper Large V3 — Luganda NSS v24 (Encoder + SpecAugment + Data Augmentation)
Stage 2 fine-tune on the CDLI Ugandan Luganda Nonstandard Speech dataset. Built on top of lg_waxal_tune_whisper_large_7 (Stage 1 with SpecAugment). This is Variant 3 of the Stage 2 experiments — encoder only with SpecAugment and data augmentation.
Training Details
- Dataset: cdli/ugandan_luganda_nonstandard_speech_v1.0
- Stage 1 checkpoint: lg_waxal_tune_whisper_large_7/best_model
- Encoder frozen: No (encoder trained only)
- Decoder frozen: Yes
- Projection frozen: Yes
- SpecAugment: On
- Data augmentation: Gaussian noise (p=0.5), gain perturbation (p=0.5), speed perturbation via torchaudio resample (p=0.5)
- Learning rate: 5e-5 (polynomial decay)
- Max steps: 1000
- Best checkpoint: Step 300
- Output dir: lg_nss_tune_whisper_large_v24
Results (checkpoint-300)
| Metric | Value |
|---|---|
| Overall WER (normalized) | 0.76 |
| Overall CER (normalized) | 0.297 |
| Avg WER (normalized) | 0.544 |
| Avg CER (normalized) | 0.181 |
| Meaning preservation | 82.6% |
WER by Severity
| Severity | Baseline WER | Fine-tuned WER |
|---|---|---|
| Mild | 0.89 | 0.49 |
| Moderate | 0.96 | 0.55 |
| Severe | 0.97 | 0.64 |
WER by Etiology
| Etiology | Baseline WER | Fine-tuned WER |
|---|---|---|
| Acquired hearing impairment | 0.98 | 0.71 |
| Cerebral Palsy | 0.95 | 0.45 |
| Cerebral palsy - cerebral malaria | 0.93 | 0.57 |
| Developmental | 0.97 | 0.56 |
| Idiopathic | 0.80 | 0.40 |
| Structural (tongue tie) | 0.96 | 0.66 |
WER by Speaker
| Speaker | Severity | Etiology | Baseline WER | Fine-tuned WER |
|---|---|---|---|---|
| UG001 | Mild | Cerebral palsy - cerebral malaria | 0.93 | 0.57 |
| UG014 | Mild | Idiopathic | 0.80 | 0.40 |
| UG022 | Mild | Developmental | 0.95 | 0.51 |
| UG021 | Moderate | Structural (tongue tie) | 0.96 | 0.66 |
| UG036 | Moderate | Cerebral Palsy | 0.95 | 0.45 |
| UG052 | Moderate | Developmental | 0.98 | 0.55 |
| UG042 | Severe | Developmental | 0.98 | 0.61 |
| UG057 | Severe | Acquired hearing impairment | 0.98 | 0.71 |
| UG058 | Severe | Developmental | 0.98 | 0.59 |
Comparison with Other Variants (Stage 2)
| Variant | Config | Avg WER | Meaning Preservation |
|---|---|---|---|
| v22 (Run 4) | Full model + SpecAugment | 0.534 | 84.7% |
| v23 (Run 5) | Encoder only + SpecAugment | 0.555 | 80.4% |
| v24 (Run 6) | Encoder only + SpecAugment + Data Aug | 0.544 | 82.6% |