Whisper-tiny Hinglish (Romanized)

A fine-tune of openai/whisper-tiny that transcribes Hindi, English, and code-switched Hinglish Indian speech directly into natural Roman script — the way people actually type Hinglish (kya scene hai, not क्या सीन है and not a stiff academic transliteration).

• 39 M parameters, 4 encoder + 4 decoder layers, 80-mel, 16 kHz — runs on CPU / edge.
• Output is always Roman, even for pure-Hindi speech. No Devanagari, no separate transliteration step at inference time.
• Built for on-device dictation for Indian users, but released here as a standalone, general-purpose model for the community.

This is a small model. It is fast and convenient, not state-of-the-art. See Limitations.

Example outputs

The model emits Roman text directly from audio. Target style (illustrative):

English spans are kept verbatim; Hindi spans come out in natural Roman spelling.

Usage

from transformers import pipeline

asr = pipeline("automatic-speech-recognition",
               model="<your-username>/whisper-tiny-hinglish",
               chunk_length_s=30)          # for clips longer than 30 s

print(asr("clip.wav")["text"])

Or with raw 16 kHz mono audio:

import soundfile as sf
audio, sr = sf.read("clip.wav")            # resample to 16 kHz mono if needed
print(asr({"raw": audio, "sampling_rate": 16000})["text"])

The model does not require a forced language token — it was trained to emit Roman for every input. No post-processing / transliteration is needed.

Training data

Fine-tuned on Indian-accent, read- and prompted-speech corpora, chosen for their heavy Hindi–English code-switching:

Utterances were filtered to ~0.5–20 s. Original transcripts are mixed Devanagari + Latin.

Romanization of the targets (the key step)

The goal is Roman output. So every training target was converted to natural Roman before fine-tuning:

• Devanagari spans → romanized with LIPI, a compact CTC Devanagari→Roman model that produces typed-Hinglish spellings (क्या → kya, है → hai, जो → jo), not academic IAST.
• English (Latin) spans in the transcripts were left unchanged.

The model then learns audio → Roman end-to-end, so LIPI is only a data-prep tool and is not needed at inference.

Strengths

• Natural romanized Hinglish output, ready to paste — no transliteration step downstream.
• Handles code-switching within a sentence (Hindi ↔ English) in one pass.
• Tiny & fast: real-time on CPU, sub-second on a modest GPU; ~150 MB fp32 (quantizes well).
• Robust to Indian accents on short, clear, conversational/dictation-style audio.

Limitations

• Small-model accuracy. Expect errors on noisy audio, fast speech, strong accents, overlapping speakers, or domain jargon.
• English spelling drift. Uncommon English words can come out phonetically (crackers → crackers/crakers), since the training signal is Indian-accented and Hindi-weighted.
• Romanization is phonetic, not canonical. Spellings follow how Hindi is typed and can vary (hai/hain, ki/kii). There is no single "correct" Roman spelling.
• Short-form. Trained on ≤ ~20 s utterances; use chunk_length_s=30 for longer audio, quality on very long recordings is untested.
• No reliable punctuation/casing. Output is lightly punctuated at best.
• Language scope. Hindi + English + Hinglish (Indian). Other Indic languages are out of scope.
• Domain. Read/prompted Indian speech; far-field, telephony, and heavy background noise are under-represented.

License

The model weights are released under the MIT License, the same as the base model openai/whisper-tiny. You can use, modify, and redistribute the model, including in commercial and free apps.

Training data

The model was fine-tuned on:

• HiACC (Hinglish Adult & Children Code-switched Corpus): CC BY 4.0, https://doi.org/10.5281/zenodo.15551669
• MUCS 2021 Hindi-English (OpenSLR SLR104): CC BY-SA 4.0, https://openslr.org/104/
• Common Voice (English): CC0

No dataset audio or transcripts are distributed with this model. MUCS is CC BY-SA 4.0. Whether ShareAlike terms carry over to model weights trained on such data is legally unsettled. If this matters for your use case, please review those terms yourself.

Citation

@misc{whisper-tiny-hinglish,
  title  = {Whisper-tiny Hinglish (Romanized)},
  author = {Abhishek Gautam},
  year   = {2026},
  note   = {Fine-tuned openai/whisper-tiny for romanized Hindi/English/Hinglish ASR;
            targets romanized with LIPI (https://github.com/ABHISHEKgauti25/lipi)}
}