Whisper Tiny Fine-Tuned on 5 English Accents
This model is a fine-tuned version of openai/whisper-tiny trained on a multi-accent English dataset for assignment EE5C04 (Foreign Accent English ASR) at Trinity College Dublin.
- Developer: Amjad Hamidi (Student ID: 26357330)
- Model Type: Automatic Speech Recognition (Seq2Seq Transformer)
- Base Model:
openai/whisper-tiny - Training Steps: 2,000 steps
Intended Use
This model is fine-tuned to transcribe multi-accented English speech, addressing severe acoustic mismatch and automatic language detection failures in pre-trained ASR systems.
Dataset & Training Details
The model was trained on a unified multi-accent English dataset comprising 9,806 training samples and 2,484 test samples across 5 assigned accents:
- Native Accents: Canadian English (
naharte/canadian_english), Indian English (naharte/indian_english) - Non-Native Accents: Spanish English (
naharte/spanish_english), Portuguese English (naharte/portuguese_english), Russian English (naharte/russian_english)
Hyperparameters
- Learning Rate: 1e-5
- Per Device Train Batch Size: 2
- Gradient Accumulation Steps: 1
- Warmup Steps: 500
- Max Steps: 2,000
- FP16 / Gradient Checkpointing: Enabled
- Optimizer: AdamW
Evaluation Results
WER Performance Comparison
| Accent Group | Accent Name | Baseline WER (Pre-trained) | Fine-tuned WER (2000 steps) |
|---|---|---|---|
| Native 1 | Canadian English | 41.04% | 17.13% |
| Native 2 | Indian English | 41.82% | 9.91% |
| Non-Native 1 | Spanish English | 126.27% | 11.59% |
| Non-Native 2 | Portuguese English | 38.81% | 11.97% |
| Non-Native 3 | Russian English | 41.29% | 10.42% |
Training Steps Summary Log
- Step 1000: Training Loss =
0.3542| Validation Loss =0.2788| WER =13.31% - Step 2000: Training Loss =
0.3213| Validation Loss =0.2493| WER =11.90%
How to Use
from transformers import WhisperForConditionalGeneration, WhisperProcessor
model_id = "Amjad-Hamidi/Whisper-tiny-5accents-Amjad"
processor = WhisperProcessor.from_pretrained(model_id)
model = WhisperForConditionalGeneration.from_pretrained(model_id)