🎙️ Vaani-1.1 Lite ASR

Next-Generation Multimodal Speech-LLM for High-Precision Hindi & English Transcription

Developed by Sorika Labs (Formerly designated as Vaani-v3)

Hugging Face Model • Training Dataset • Sorika Labs Organization

---

🌟 Overview & Key Breakthroughs

Vaani-1.1 Lite ASR is Sorika Labs' flagship multimodal Speech-LLM foundation model, fine-tuned on top of Qwen/Qwen3-ASR-0.6B-hf. It combines an 18-layer Conformer audio tower with a 28-layer Qwen3 LLM text decoder.

In comprehensive multi-model benchmarking across 10 standardized datasets, Vaani-1.1 Lite achieved the #1 lowest Hindi Average WER (11.78%), outperforming Alibaba's base Qwen3-ASR, OpenAI Whisper-Small, and Whisper-Turbo.

Key Highlights:

• 🥇 #1 SOTA on Indian Speech (11.78% Hindi Avg WER):
  • Kathbath Hindi: 12.36% WER (Beats Whisper-Small by 37.64% absolute!)
  • FLEURS Hindi: 14.64% WER
  • Private Conversational Hindi: 8.33% WER
• 🥇 #1 on Noisy English Speech:
  • LibriSpeech Other: 3.60% WER (Outperformed all Whisper models and Base Qwen3)
• 🛡️ Devanagari Script Purity: Native Unicode NFC normalization eliminates phonetic English/Romanized confusion common in traditional models.
• 📦 Standalone Merged Weights: LoRA adapters are 100% merged into model.safetensors (~1.2 GB), requiring zero adapter overhead or secondary downloads.

---

🧭 Sorika Labs Vaani Speech Series Portfolio

The Vaani family is engineered by Sorika Labs to provide production-ready speech intelligence across diverse computational constraints:

---

📊 Grand Benchmark Leaderboard (10 Standardized Benchmarks)

All models were evaluated under identical real-world conditions across 7 English benchmarks (meeting, financial, podcast, clean read, noisy speech, parliamentary) and 3 Indic/Hindi benchmarks:

Lower WER indicates superior recognition accuracy. Speed is measured in Real-Time Factor (RTFx) on NVIDIA GPU.

---

🚀 Quickstart & Inference

Vaani-1.1 Lite utilizes the modern Multimodal Speech-LLM architecture. Transcribe audio files directly with transformers:

import torch
from transformers import AutoProcessor, AutoModelForMultimodalLM

MODEL_ID = "sorika-labs/vaani1.1-lite-asr"

# Load processor and model
processor = AutoProcessor.from_pretrained(MODEL_ID)
model = AutoModelForMultimodalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Prepare transcription request (16kHz mono recommended)
inputs = processor.apply_transcription_request(audio="sample_audio.wav").to(model.device, model.dtype)

# Generate transcription
with torch.no_grad():
    output_ids = model.generate(**inputs, max_new_tokens=256)

# Extract generated tokens
generated_ids = output_ids[:, inputs["input_ids"].shape[1]:]
transcription = processor.decode(generated_ids, return_format="transcription_only")

print("Transcription:", transcription[0])

---

📊 Training Recipe & Convergence

• Dataset: sorika-labs/vaani-asr (Curated 60% Hindi / 40% English mix)
• Optimization: High-rank LoRA ($r=64, \alpha=128$) across all attention and MLP projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj).
• Shielding: Prompt audio and formatting tokens masked with labels = -100 to prevent language drift.
• Convergence: Validation loss reached an optimal 0.8984 cross-entropy (Perplexity ~2.4) with zero catastrophic forgetting.

---

⚙️ Architecture Specifications

• Base Backbone: Qwen/Qwen3-ASR-0.6B-hf (~600M Parameters)
• Audio Encoder: 18-layer Conformer (186M parameters, 128 mel filterbanks, 8x time downsampling)
• Language Decoder: 28-layer Qwen3 LLM (Hidden size 1024, Grouped Query Attention 16/8 heads)
• Context Length: 8,192 tokens
• License: Apache 2.0

---

🏢 Developed by Sorika Labs

Sorika Labs — Advancing open-source speech AI for Indian and global languages.