Whisper Ugandan English Dysarthric Speech Recognition (v8)

A fine-tuned Whisper-Small model for Ugandan English speakers with dysarthria, using LoRA (Low-Rank Adaptation) for parameter-efficient fine-tuning.

Model Description

• Base model: openai/whisper-small
• Fine-tuning method: LoRA (r=128, alpha=256)
• Fine-tuned on: cdli/ugandan_english_nonstandard_speech_v1.0
• Language: Ugandan English (en)

Performance

Training Data

• Real dysarthric samples: 5,172
• Augmented samples: 46,000 (16 medical dysarthric augmentation types)
• Total: 51,172 samples

LoRA Configuration

LoraConfig(
    r=128,
    lora_alpha=256,
    target_modules=["q_proj","k_proj","v_proj","o_proj","fc1","fc2",
                    "encoder_attn.q_proj","encoder_attn.k_proj",
                    "encoder_attn.v_proj","encoder_attn.out_proj"],
    lora_dropout=0.1,
    bias="none"
)

How to Use

from transformers import WhisperProcessor, WhisperForConditionalGeneration
from peft import PeftModel
import torch

# Load base model + LoRA adapter
base_model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small")
model = PeftModel.from_pretrained(base_model, "nugwa-mark/whisper-ugandan-english-dysarthric-v8")
processor = WhisperProcessor.from_pretrained("nugwa-mark/whisper-ugandan-english-dysarthric-v8")

import librosa
audio, sr = librosa.load("your_audio.wav", sr=16000)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    generated = model.generate(
        inputs.input_features,
        num_beams=10,
        repetition_penalty=1.1,
    )
transcription = processor.decode(generated[0], skip_special_tokens=True)
print(transcription)

Offline Usage

# Save locally
processor.save_pretrained("./english-asr-offline")
model.save_pretrained("./english-asr-offline")

Intended Use

Same as Luganda model — assistive technology for Ugandan dysarthric speakers in healthcare, education, and daily communication contexts.

Citation

@misc{ugandan-english-dysarthric-asr-2026,
  title={Ugandan English Dysarthric Speech Recognition},
  author={Nugwa Mark},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/nugwa-mark/whisper-ugandan-english-dysarthric-v8}
}