Whisper Small Manglish
Fine-tuned Whisper Small model for Malayalam-English code-switched automatic speech recognition (Manglish).
Results
| Model | WER ↓ | CER ↓ |
|---|---|---|
| Whisper Medium | 111.64% | 82.64% |
| Sarvam Saaras v3 Codemix | 48.61% | 21.95% |
| Whisper Small (Fine-Tuned) | 44.38% | 16.53% |
Training Data
Manglish ASR Benchmark
- Train: 2,098 clips
- Validation: 285 clips
- Test: 500 clips
Data Leakage Verification
- Train ∩ Test = 0
- Validation ∩ Test = 0
- Train ∩ Validation = 0
Dataset
https://huggingface.co/datasets/jona276/manglish-asr-benchmark
Usage
from transformers import pipeline
pipe = pipeline( "automatic-speech-recognition", model="jona276/whisper-small-manglish" )
result = pipe("audio.wav") print(result["text"])
Limitations
- Evaluated only on the Manglish benchmark
- Strict transcript matching
- Script-equivalent words are counted as errors