whisper-small-ur-en
A bilingual Urdu–English fine-tune of openai/whisper-small, trained for accurate transcription of Pakistani Urdu and American English speech.
Results
Evaluated on 200 held-out clips per language:
| Language | Baseline (openai/whisper-small) WER |
Fine-tuned WER |
|---|---|---|
| Urdu | 43.77% | 29.38% |
| English | 26.89% | 8.50% |
Training data
- 4,711 clips: Fleurs
ur_pk+en_us, with per-sample language tokens - 200 test clips per language, held out from training
Usage
from transformers import WhisperProcessor, WhisperForConditionalGeneration
processor = WhisperProcessor.from_pretrained("openai/whisper-small")
model = WhisperForConditionalGeneration.from_pretrained("ubaidngt/whisper-small-ur-en")
Limitations
- Short-form inference: audio is truncated to the first 30 seconds (Whisper's native window).
- Trained mostly on read speech (Fleurs); noisy or heavily accented conversational speech may degrade accuracy.
- Urdu gains are real but smaller than English — further data (Pakistani conversational Urdu) is the planned next step.
License
Apache-2.0