PakUrdu-Conversational-ASR — Whisper Large-v3 Turbo Urdu Call Fine-tune (LoRA)

LoRA adapter (r=16, α=32, attention projections) on top of openai/whisper-large-v3-turbo (immutable revision 41f01f3fe87f28c78e2fbf8b568835947dd65ed9), fine-tuned for telephone-domain conversational Urdu with a two-stage curriculum — broad public-domain Urdu speech first, then specialization on real customer-call audio from SimpleRishta (سمپل رشتہ), PII-masked, speaker-safe splits. No SimpleRishta audio or transcripts are published; only these adapter weights. This release (r14) supersedes r12 (8.9%) on the same frozen evaluation.

Live demo: PakUrdu-ASR-Demo — runs fully in your browser (WebGPU); your audio never leaves your device.

Full quality, no GPU of your own: open inference_notebook.ipynb in Google Colab or Kaggle — a free cloud GPU runs the complete unquantized model on your clip.

✅ Verified end-to-end — the 7.96% headline number is reproduced live in a public Kaggle proof notebook: e1-sr-proof-r14-sr133 (full details under Results).

Intended / out-of-scope use

• Intended: transcribing Urdu telephone-call audio (customer-call traffic, conversational telephony speech), decoded with language=ur, task=transcribe, greedy.
• Out of scope: general-purpose studio/broadcast Urdu (our internal measurements place it behind general-purpose fine-tunes there), languages other than Urdu, speaker identification, or any decision-making about individuals.

Results

133 held-out SimpleRishta calls, zero caller overlap with training, corpus C0 WER (normalized, PII-masked), identical decode settings for both models:

Reference transcription: an independent high-accuracy Whisper-family model (eval-only, never trained on). Both models scored identically against it. Head-to-head on the same calls vs our previous in-house baseline (17.3%): this adapter wins 120 / loses 9 / ties 4. Zero degenerate generations across all stability checks (repetition-rate and length-ratio guards, clean greedy decoding). Aggregate protocol and numbers: eval/aggregate-results.json.

Independent reproduction (live run): the 7.96% figure is re-derived end-to-end on a clean GPU in this public Kaggle notebook — model loaded, 133 calls decoded, fresh WER printed against the certified value (observed drift 0.0000): Kaggle proof notebook: e1-sr-proof-r14-sr133.

Usage

import torch, soundfile as sf
from peft import PeftModel
from transformers import WhisperForConditionalGeneration, WhisperProcessor

base = "openai/whisper-large-v3-turbo"
processor = WhisperProcessor.from_pretrained(base)
model = WhisperForConditionalGeneration.from_pretrained(base)
model = PeftModel.from_pretrained(model, "KhiredNetworks/PakUrdu-Conversational-ASR").merge_and_unload()

speech, sr = sf.read("call.wav", dtype="float32")
inputs = processor(speech, sampling_rate=sr, return_tensors="pt")
ids = model.generate(inputs.input_features, language="ur",
                     task="transcribe", do_sample=False)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

A runnable version ships as inference_example.py. To run the complete model on a free cloud GPU — no GPU of your own, nothing installed — open inference_notebook.ipynb (works in Google Colab and Kaggle).

Training data & governance

• Fine-tuned on real SimpleRishta customer-call audio (16 kHz telephone Urdu; transcripts PII-masked before training; brand name preserved; caller-group-safe train/dev split — no caller appears in both).
• Evaluation set frozen and never used in training.
• Training method: LoRA (PEFT), two-stage curriculum (stage 1: broad public-domain Urdu speech, one epoch; stage 2: telephony specialization with public-data replay, one epoch), fresh from the pinned base revision — no continuation from any other checkpoint.
• No SimpleRishta audio, transcripts, or caller-identifying data are published. Published results are aggregate-only.
• Full training/eval audit trail retained internally (sealed compute records; certified numbers independently reproduced end-to-end on a clean GPU run, drift 0.0000).

Transcription convention & inference settings

Urdu script output; evaluation text normalized under a corpus C0 protocol (Unicode normalization, digit/punctuation folding, brand canonicalization, PII masking) before scoring. Decode: greedy (do_sample=False), language=ur, task=transcribe, 16 kHz mono input. FP16 on GPU / FP32 on CPU both supported; results certified on a single T4.

Limitations

• Scores are vs a model reference, not human transcription.
• Evaluated on accepted (clean) call audio; performance on heavily noisy or overlapped speech is not characterized here.
• Urdu only (language=ur fixed in the intended decode config).

License, attribution & contact

• Code/adapter: Apache-2.0 (see LICENSE). Base model: openai/whisper-large-v3-turbo — Whisper license and attribution chain apply downstream.
• Citation: see CITATION.cff (Khired Networks).
• Issues: use the repo's Issues tab.