whisper-large-v3-turbo-hinglish

A fine-tune of openai/whisper-large-v3-turbo for Hinglish — code-switched Hindi–English — automatic speech recognition (ASR). The model transcribes speech that mixes Hindi and English within the same utterance (romanized English tokens kept as English, Hindi in Devanagari), a setting where the base model tends to struggle.

The LoRA adapter has been merged into the base, so this is a complete, standalone Whisper model — load and use it exactly like any WhisperForConditionalGeneration checkpoint.

Usage

from transformers import pipeline

asr = pipeline("automatic-speech-recognition",
               model="Deepro713/whisper-large-v3-turbo-hinglish")

print(asr("your_audio.wav")["text"])

Or with the model + processor directly:

import torch, librosa
from transformers import WhisperForConditionalGeneration, WhisperProcessor

repo = "Deepro713/whisper-large-v3-turbo-hinglish"
model = WhisperForConditionalGeneration.from_pretrained(repo, torch_dtype=torch.float16).to("cuda")
proc  = WhisperProcessor.from_pretrained(repo)

wav, _ = librosa.load("your_audio.wav", sr=16000, mono=True)
feat = proc.feature_extractor(wav, sampling_rate=16000, return_tensors="pt").input_features.to("cuda", torch.float16)
ids  = model.generate(feat, language="english", task="transcribe", max_new_tokens=128)
print(proc.tokenizer.decode(ids[0], skip_special_tokens=True))

Audio should be 16 kHz mono; clips ≤ 30 s.

Training


Base model	`openai/whisper-large-v3-turbo` (809 M params)
Method	LoRA (r=16, α=32, dropout=0.05) on the attention `q_proj`/`v_proj`, then merged
Data	`agarwalayushi/hinglish` — 2,000-clip subset (1,602 train / 200 val / 201 test)
Preprocessing	resampled to 16 kHz mono, loudness-normalized, silence-trimmed, ≤ 30 s segments
Schedule	3 epochs, batch size 1, lr 1e-4, bf16, gradient checkpointing
Hardware	1× NVIDIA GB10 (Grace-Blackwell), unified memory, resource-capped

Evaluation

Word Error Rate (WER, lower is better) on a 40-clip held-out sample of the dataset's test split (the model never saw these during training):

Model	WER
`openai/whisper-large-v3-turbo` (base)	1.224
This model (fine-tuned)	0.306

The base model's very high WER reflects how poorly it handles this specific code-switched, casual, tech-tutorial-style Hinglish — it is not a statement about Whisper's general Hindi ability. The fine-tune tracks the domain far more closely.

Limitations & intended use

Small, single-domain training set. The corpus is casual/tech-tutorial Banarasi-flavoured Hinglish; the model may not generalize to other accents, domains, or recording conditions, and can hallucinate on out-of-distribution audio.
Small evaluation sample (40 clips) — treat the WER as indicative, not a rigorous benchmark.
Best for research, experimentation, and code-switched Hindi–English transcription in a similar domain. Not validated for high-stakes or production use.
Inherits Whisper's general limitations (hallucination on silence/noise, repetition, timestamp drift).

License & attribution

Model weights: MIT (inherited from the base openai/whisper-large-v3-turbo).
Training data: agarwalayushi/hinglish, licensed CC BY 4.0 — attribution to its author is required and given here.

Citation

@misc{whisper_large_v3_turbo_hinglish,
  title  = {whisper-large-v3-turbo-hinglish},
  author = {Deepro713},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Deepro713/whisper-large-v3-turbo-hinglish}},
  note   = {Fine-tune of openai/whisper-large-v3-turbo on agarwalayushi/hinglish (CC BY 4.0)}
}