Whisper Small - Ugandan English v11 (Speech Impairment Optimized)

Model Description

Fine-tuned version of OpenAI Whisper Small optimised for Ugandan English with speech impairment robustness for assistive technology applications.

Performance

Training Data

• 20,834 clean Ugandan court recordings (2-10 seconds, 5-20 words)
• 83,336 total samples after 4x speech impairment augmentation
• 13 Ugandan English speakers (9 male, 4 female)

Impairment Types Simulated

1. Dysarthria - slowed rate, tremor, imprecise consonants (stroke, CP, MS, ALS)
2. Dysphonia breathy - aspiration noise, pitch jitter (vocal cord paralysis, Parkinson's)
3. Dysphonia hoarse - harmonic distortion, diplophonia (nodules, cord damage)
4. Apraxia - inconsistent rate, groping pauses (stroke, TBI)
5. Electrolarynx - robotic buzz carrier (post-laryngectomy)
6. Fluency disorders - prolongations, blocks (stuttering, cluttering)
7. Hypophonia - very soft voice, mic noise floor (Parkinson's, ALS, fatigue)
8. Device noise - bandpass, hum, dropouts (AAC devices, speaking valves)

Training Config

• Base model: openai/whisper-small
• Learning rate: 1e-5
• Effective batch size: 16 (batch 8 x grad accum 2)
• Max steps: 5,000 | Eval every 500 steps
• FP16 enabled

Use Cases

• Assistive technology for people with speech disabilities
• Court transcription for Ugandan English speakers
• Accessibility applications for dysarthria, stuttering, apraxia
• Medical dictation for patients with motor speech disorders

How to Use

from transformers import WhisperProcessor, WhisperForConditionalGeneration
import torch, librosa

processor = WhisperProcessor.from_pretrained('Nkugwa/whisper-small-ugandan-v11')
model = WhisperForConditionalGeneration.from_pretrained('Nkugwa/whisper-small-ugandan-v11')
model = model.to('cuda' if torch.cuda.is_available() else 'cpu')

audio, sr = librosa.load('your_audio.wav', sr=16000)
inputs = processor(audio, sampling_rate=16000, return_tensors='pt').input_features
with torch.no_grad():
    generated = model.generate(inputs, language='english', task='transcribe')
text = processor.batch_decode(generated, skip_special_tokens=True)[0]
print(text)

Limitations

• Augmentation is synthetic — real impaired speakers may differ
• Optimised for Ugandan English; other accents may see reduced accuracy
• Test thoroughly with real users before deploying in clinical contexts