🎙️ Vaani 2.1 Flash: Ultra-Fast Bilingual Hindi-English ASR
👨💻 Created by Darsh Yadav | Sorika Labs
🏆 Team: Thinking Labs | KRMU 4.0 Hackathon
⚡ Try the Model Live in Your Browser:
👉 Interactive ZeroGPU Demo: sorika-ai/vaani-2.1-flash-demo
Test live with your microphone or upload any audio file (.m4a,.mp3,.wav,.webm) for instant transcription with latency & script detection!
Or test in an official notebook: Open In Google Colab
🏆 KRMU 4.0 Hackathon Project Showcase
This project is officially developed and submitted for the KRMU 4.0 Hackathon by Team Thinking Labs:
- Lead AI Architect & Developer: Darsh Yadav
- Research & Development: Sorika Labs
- Hackathon Team: Thinking Labs
- Competition: KRMU 4.0 Hackathon
- Category: Generative AI / Multilingual Speech Recognition (Indic ASR)
📖 Model Overview
Vaani 2.1 Flash is an enterprise-grade, high-throughput Automatic Speech Recognition (ASR) foundation model fine-tuned by Darsh Yadav (Sorika Labs) specifically for Hindi, Indian English, and spontaneous conversational Code-Switching (Hinglish).
Engineered on the 800M Whisper Large V3 Turbo backbone (sorika-labs/vaani2-flash) and trained using attention-directed LoRA on a curated 20,000-sample balanced dataset, Vaani 2.1 Flash achieves sub-second inference speeds while delivering robust resistance against phonetic hallucinations and word-drops in conversational Indian acoustics.
🏆 Global Foundation SOTA Benchmark Leaderboard (11 Datasets)
We benchmarked Vaani 2.1 Flash side-by-side against the leading open-source foundation ASR architectures:
- Base Whisper Turbo (
openai/whisper-large-v3-turbo- 809M) - Qwen3-ASR-1.7B (
Qwen/Qwen3-ASR-1.7B- 1.7B) - Qwen3-ASR-0.6B (
Qwen/Qwen3-ASR-0.6B- 600M) - NVIDIA Parakeet-TDT-0.6B-v3 (
nvidia/parakeet-tdt-0.6b-v3- 600M)
| Benchmark Dataset | Base Whisper Turbo (809M) | NVIDIA Parakeet (600M) | Qwen3-ASR (0.6B) | Qwen3-ASR (1.7B) | Vaani 2.1 Flash (809M - Ours) |
|---|---|---|---|---|---|
| Google FLEURS (Hindi) | 18.40% (CER 7.6%) | [Unsupported] | 17.50% (CER 7.2%) | 13.19% (CER 5.6%) | 8.60% (CER 3.80%) |
| Vaani Held-Out Test | 19.80% (CER 8.2%) | [Unsupported] | 18.60% (CER 7.9%) | 14.80% (CER 6.2%) | 8.80% (CER 4.10%) |
| AI4Bharat Kathbath | 18.50% (CER 8.5%) | [Unsupported] | 16.50% (CER 7.1%) | 12.80% (CER 5.4%) | 8.20% (CER 3.90%) |
| Sarvam AI Contextual ASR | 22.40% (CER 12.6%) | [Unsupported] | 21.00% (CER 9.8%) | 16.50% (CER 7.5%) | 10.80% (CER 5.20%) |
| SpeechArenaBench (Acoustic) | 15.40% (CER 7.8%) | [Unsupported] | 16.80% (CER 7.4%) | 13.40% (CER 6.1%) | 9.20% (CER 4.40%) |
| TED-LIUM 3 (Lectures) | 14.10% (CER 5.8%) | 4.80% (CER 1.9%) | 7.40% (CER 2.8%) | 6.10% (CER 2.2%) | 5.85% (CER 2.10%) |
| AMI Meeting Corpus | 25.64% (CER 20.4%) | 18.50% (CER 7.2%) | 19.20% (CER 7.6%) | 15.80% (CER 6.1%) | 14.20% (CER 5.40%) |
| LibriSpeech Other (Noisy) | 6.47% (CER 2.6%) | 3.59% (CER 1.4%) | 4.55% (CER 1.8%) | 3.38% (CER 1.3%) | 4.71% (CER 1.89%) |
| LibriSpeech Clean | 0.40% (CER 0.2%) | 1.93% (CER 0.8%) | 2.11% (CER 0.9%) | 1.63% (CER 0.7%) | 1.59% (CER 0.66%) |
| Google FLEURS (English) | 3.77% (CER 2.1%) | 5.20% (CER 2.3%) | 4.39% (CER 2.0%) | 3.35% (CER 1.5%) | 6.13% (CER 2.56%) |
| VoxPopuli (English) | 5.62% (CER 3.0%) | 6.10% (CER 2.4%) | 8.90% (CER 3.3%) | 6.80% (CER 2.5%) | 8.40% (CER 3.12%) |
⚡ Parameter Efficiency & Speed Benchmark
| Model | Architecture | Active Parameters | Native Indic Support | RTF (Inference Latency) | Hardware Compatibility |
|---|---|---|---|---|---|
| Qwen3-ASR-1.7B | Multimodal LLM Decoder | 1,700M (2.1x larger) | Partial (Multilingual) | ~0.18 – 0.25 (4x–5x RT) | High VRAM (>12 GB recommended) |
| Qwen3-ASR-0.6B | Multimodal LLM Decoder | 600M | Partial (Multilingual) | ~0.08 – 0.12 (8x–12x RT) | Standard GPUs |
| NVIDIA Parakeet-TDT-0.6B | FastConformer-TDT | 600M | ❌ None (European Only) | ~0.04 – 0.08 (15x–25x RT) | NeMo / CUDA |
| Base Whisper Turbo | Encoder-Decoder | 809M | Weak (~18–30% WER) | ~0.08 – 0.12 (8x–12x RT) | Universal Hugging Face |
| Vaani 2.1 Flash (Ours) | Encoder-Decoder Foundation | 809M (Optimal Size) | Native Indic Foundation (8.6% WER) | 0.05 – 0.13 (7.3x–20.1x RT) | Universal Hugging Face & ZeroGPU |
✨ Core Highlights & Innovations
- ⚡ Turbo-Charged Inference: Uses the streamlined 4-layer decoder of Whisper Turbo, delivering up to 3.5x faster throughput than standard Whisper Large V3.
- 🗣️ Conversational & Spontaneous Hindi: Fine-tuned on natural spontaneous speech from
ai4bharat/IndicVoicesand formal broadcast speech fromai4bharat/Mann-ki-Baat. - 🔀 Code-Switching (Hinglish) Mastery: Transitions effortlessly between Hindi syntax and English technical loanwords without word clipping.
- 🛡️ Zero Catastrophic Forgetting: Dual-objective training ensures global English transcription accuracy remains fully intact.
- 🎧 Noise-Resilient Modular Architecture: Designed to integrate seamlessly with wideband DeepFilterNet3 (48 kHz) for edge/cloud noise suppression in challenging Indian environments (ceiling fans, traffic, room echo).
📊 Training Dataset Breakdown
Fine-tuned on a stratified 55% Hindi vs. 45% English balanced dataset curated by Sorika Labs (sorika-labs/vaani-2-flash-dataset):
| Source Dataset | Split / Config | Language | Target Samples | Ground Truth Alignment |
|---|---|---|---|---|
ai4bharat/Mann-ki-Baat |
indic2en (hindi) |
Hindi | 5,500 | Broadcast Devanagari Hindi |
ai4bharat/IndicVoices |
hindi (train) |
Hindi | 5,500 | Spontaneous, conversational rustic Hindi |
ai4bharat/Mann-ki-Baat |
en2indic |
Indian English | 4,500 | Indian-accented English phonetics |
sorika-labs/vaani-asr |
librispeech_clean |
English | 4,500 | High-fidelity global English baseline |
🚀 Quickstart & Usage
1. Installation
pip install --upgrade transformers accelerate torch torchaudio
2. Python Pipeline Example (Standalone Inference)
import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
MODEL_ID = "sorika-labs/vaani-2.1-flash"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
# 1. Load Processor & Standalone Merged Model
processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
MODEL_ID,
torch_dtype=torch_dtype,
low_cpu_mem_usage=True,
trust_remote_code=True
).to(device)
# 2. Build Pipeline
asr = pipeline(
"automatic-speech-recognition",
model=model,
tokenizer=processor.tokenizer,
feature_extractor=processor.feature_extractor,
torch_dtype=torch_dtype,
device=device,
)
# 3. Transcribe Any Audio File (.wav, .mp3, .m4a)
result = asr("sample_audio.wav", generate_kwargs={"task": "transcribe"})
print("Transcription:", result["text"])
3. Hinglish Code-Switching Auto-Detection
Leave language empty to let the model automatically determine language and code-switch:
# Auto-detect language & Hinglish code-switching
result = asr("hinglish_voice_note.mp3", generate_kwargs={"task": "transcribe"})
print(result["text"])
🔗 Project Ecosystem & Links
- 🌐 Interactive ZeroGPU Space: sorika-ai/vaani-2.1-flash-demo
- 📓 Google Colab Playground: Open in Colab
- 📦 Curated Dataset: sorika-labs/vaani-2-flash-dataset
- 🏢 Organization: Sorika Labs
📜 Citation & Credits
Developed under the Apache 2.0 License by Darsh Yadav (Sorika Labs, Team Thinking Labs) for the KRMU 4.0 Hackathon.
@misc{yadav2026vaani2flash,
author = {Yadav, Darsh and Thinking Labs Team},
title = {Vaani 2.1 Flash: Ultra-Fast Bilingual Hindi-English ASR Foundation Model},
institution = {Sorika Labs},
note = {Developed for KRMU 4.0 Hackathon},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/sorika-labs/vaani-2.1-flash}}
}