whisper-small-accented-en

Fine-tuned openai/whisper-small for transcribing non-native (accented) English. On a speaker-disjoint held-out test set it reduces word error rate (WER) from 21.30% to 15.53% (a 5.77-point absolute, ~27% relative improvement) over the base model.

Results

WER measured with jiwer against gold transcripts on a speaker-disjoint held-out split (no test speaker appears in training).

Intended use

Transcription of read/prepared English speech from non-native speakers. It is a research/portfolio model and is not deployed in any production system.

How to use

from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="katherineahn/whisper-small-accented-en",
)
print(asr("audio.wav")["text"])

Training data

A subset of facebook/voxpopuli, config en_accented: European Parliament recordings from non-native English speakers spanning 15 first-language backgrounds. Filtered to examples with gold transcripts, then split speaker-disjoint (364 speakers, 54 held out) and capped at 3,000 training / 400 test clips. A small number of clips that failed to decode were skipped.

Training procedure

Fine-tuned with the Hugging Face Seq2SeqTrainer on a single Tesla T4 GPU.

• Base model: openai/whisper-small (~242M parameters)
• Steps: 600 (~14.5 minutes)
• Training loss: ~0.96 -> ~0.25

Limitations

• whisper-small is a small model; larger Whisper variants will transcribe more accurately at the cost of speed and memory.
• Training data is read parliamentary speech, so accuracy may drop on spontaneous, conversational, or noisy audio.
• Evaluated on a 400-clip held-out set; treat the reported WER as an estimate.

Citation

Base model: Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision (Whisper). Dataset: Wang et al., VoxPopuli.