Whisper large-v3-turbo — Persian
openai/whisper-large-v3-turbo fine-tuned for Persian speech-to-text on ~534 h of Persian speech: read speech, YouTube talks and movie dialogue.
Results
Greedy decoding with Persian forced. WER and CER are computed after normalizing both reference and hypothesis: ي/ی, ك/ک, ZWNJ, diacritics, punctuation and tatweel are removed. "CER (no spaces)" also ignores spacing. Persian often writes the same word with or without a space (میشیم / می شیم), and plain WER counts those variants as errors.
| Test set | Clips | WER base → fine-tuned | CER (no spaces) base → fine-tuned |
|---|---|---|---|
| Common Voice 17 fa test (speakers unseen in training) | 10,558 | 50.2% → 19.4% | 23.6% → 4.2% |
| YouTube, 82 videos fully held out | 1,326 | 45.3% → 20.2% | 24.4% → 10.0% |
| Filimo test (5k sample, ~2 s movie dialogue) | 5,000 | 86.5% → 31.2% | 50.4% → 16.8% |
- Exact transcripts (after normalization): Common Voice 13% → 46%, YouTube 1% → 23%, Filimo 6% → 36%.
- Runaway outputs (more than twice the reference length) dropped: YouTube 2.3% → 0.75%, Filimo 1.6% → 0.10%.
- Checkpoint choice: this is the final checkpoint (step 7,500). On a 5k-clip validation sample it slightly beat the best intermediate checkpoint (mean WER 22.8% vs 23.0%). The test sets weren't used for selection.
Usage
import torch
from transformers import pipeline
asr = pipeline("automatic-speech-recognition", model="MohammadGholizadeh/whisper-large-v3-turbo-persian",
dtype=torch.bfloat16, device="cuda")
gen = {"language": "fa", "task": "transcribe"}
print(asr("clip.wav", generate_kwargs=gen)["text"]) # file input needs ffmpeg installed
import soundfile as sf # or pass audio yourself, no ffmpeg needed
audio, sr = sf.read("clip.wav", dtype="float32")
print(asr({"raw": audio, "sampling_rate": sr}, generate_kwargs=gen)["text"])
Audio (mono) is resampled to 16 kHz automatically. The model was trained on clips of 0.5–20 s. For longer recordings pass chunk_length_s=30 to the pipeline.
Training
- Data: 479k (audio, transcription) pairs, ~534 h. Filimo train (322k), YouTube (128k, held-out videos excluded), Common Voice 17 fa train (29k). Resampled to 16 kHz; tatweel speaker-turn marks removed from targets.
- Optimization: 1 epoch (7,500 steps, batch 64), AdamW, lr 1e-5 with linear decay and 500 warmup steps. bf16 autocast over fp32 weights, gradient checkpointing. About 3.9 h on one GH200.
- Scope: full fine-tune of encoder and decoder. No timestamps, no SpecAugment.
Limitations
- Trained without timestamps, so timestamp prediction wasn't trained and may be worse than the base model's.
- Spelling and spacing follow the training transcripts, which mix conventions (e.g. ZWNJ use).
- Only tested on Persian; other languages may have degraded.
- Noisy telephone audio and strong regional dialects weren't evaluated.
Code: https://github.com/msghik/embeddinggemma2-speech-retrieval