Whisper-large-v3 fine-tuned for Tatar (run 1)

openai/whisper-large-v3, fully fine-tuned on Common Voice Scripted Speech 27.0 Tatar (CC0). It writes Tatar in Cyrillic, with capital letters and punctuation.

For transcription, use run 2 instead: same recipe, trained on 165 speakers instead of 9, and 16% fewer errors. This model is kept for one reason: it never saw the official Common Voice test split, so it is the one to use for comparisons on that split.

• Write-up, with a comparison against existing Tatar ASR models: ifs.name/post/tatar-asr
• Demo: ifs/tatar-asr

Results

Error rates in percent. CER and WER are computed after lowercasing and removing punctuation; raw CER compares the output exactly as produced.

For comparison, stock Whisper-large-v3 gets 33.0% CER on the official test split. The full comparison is in the write-up.

Usage

import torch
from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="ifs/whisper-large-v3-tt-cv27-run1",
    dtype=torch.float16,   # torch.float32 on a CPU
    device="cuda",         # or "cpu"
)
result = asr(
    "audio.mp3",
    return_timestamps=True,  # needed for audio longer than 30 seconds
    generate_kwargs={
        "language": "tatar",
        "task": "transcribe",
        "num_beams": 1,
        "no_repeat_ngram_size": 3,
        "repetition_penalty": 1.2,
    },
)
print(result["text"])

The decoding settings are the ones used in evaluation; they follow BuzzASR. The weights are stored in float32 (6.2 GB); float16 halves the memory and works well for inference.

Training data

All clips come from the validated bucket of Common Voice Scripted Speech 27.0 Tatar. This run kept the official test split untouched, which leaves few speakers for everything else:

One speaker accounts for 58% of the training clips. The exact splits are in the splits/ folder of this repository. Labels keep their original casing and punctuation.

Training procedure

Full fine-tuning following the BuzzASR recipe, with finetune.py --max-dev-speaker-clips 350:

• learning rate 6.48e-6 for the decoder, 0.3 times that for the encoder
• cosine schedule, 150 warm-up steps, weight decay 4e-5
• effective batch size 32 (4 clips × 8 gradient accumulation steps)
• bf16 mixed precision, with the weights loaded in float32
• evaluation on dev CER, early stopping with patience 3, best checkpoint kept
• seed 42

Hardware: one NVIDIA A100 80GB on RunPod. Software: PyTorch 2.8.0, Transformers 5.18.0 (full environment in the write-up).

Limitations

• Trained on only 9 speakers, one of whom contributes 58% of the clips.
• Evaluated on read speech only. Accuracy on spontaneous speech, such as YouTube videos or podcasts, is untested and likely lower.
• Code-switching into Russian and timestamp accuracy are untested.

Licence

MIT, like the base model. The training data is CC0.

Author

Ilnar Salimzianov