🎙️ Vaani-1 Lite ASR

Ultra-Lightweight Multilingual Speech Recognition for Edge, Mobile & CPU Deployments

Developed by Sorika Labs

Interactive Colab Demo • Faster-Whisper Version • Sorika Labs Organization

---

🌟 Overview & Key Breakthroughs

Vaani-1 Lite ASR is an ultra-compact, high-efficiency speech recognition foundation model developed by Sorika Labs, fine-tuned on top of OpenAI's whisper-small (244M parameters).

While baseline Whisper-Small fails severely on Indian speech patterns (47.77% Hindi Average WER), Vaani-1 Lite slashes Hindi error rates down to 19.32% WER (over 2.5x error reduction) while preserving full multilingual capability across 99+ languages.

Highlights:

• 📉 Massive Hindi Error Reduction: Slashes Kathbath error rate from 50.00% down to 16.29% and FLEURS from 48.88% down to 25.56%.
• 🛡️ Zero Hallucination Loops: Fixes the notorious repeat-token looping bug common in small Whisper models.
• ⚡ Ultra-Low Memory Footprint: Requires less than 1.5 GB VRAM in FP16 and runs seamlessly on CPU, Apple Silicon, and edge devices.
• 🧩 100% Drop-In Compatible: Direct drop-in replacement for OpenAI's whisper-small in any Hugging Face, Faster-Whisper, or CTranslate2 pipeline.

---

🧭 Sorika Labs Vaani Speech Series Portfolio

The Vaani family is engineered by Sorika Labs to provide production-ready speech intelligence across diverse computational constraints:

---

📊 Grand Benchmark Leaderboard (10 Standardized Benchmarks)

All models were evaluated under identical real-world conditions across 7 English benchmarks (meeting, financial, podcast, clean read, noisy speech, parliamentary) and 3 Indic/Hindi benchmarks:

Lower WER indicates superior recognition accuracy. Speed is measured in Real-Time Factor (RTFx) on NVIDIA GPU.

---

🚀 Quickstart & Inference

1. Using Hugging Face pipeline (Recommended)

import torch
from transformers import pipeline

# Load pipeline (Auto-detects GPU or CPU)
device = "cuda:0" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if torch.cuda.is_available() else torch.float32

pipe = pipeline(
    "automatic-speech-recognition",
    model="sorika-labs/vaani1-lite-asr",
    torch_dtype=dtype,
    device=device
)

# 1. Automatic Language Detection & Transcription:
result = pipe("sample_audio.wav", generate_kwargs={"task": "transcribe"})
print("Transcription:", result["text"].strip())

# 2. Explicit Hindi Transcription (Optimized Devanagari):
result_hi = pipe("hindi_audio.wav", generate_kwargs={"language": "hindi", "task": "transcribe"})
print("Hindi:", result_hi["text"].strip())

2. Using AutoModelForSpeechSeq2Seq & AutoProcessor

import torch
import torchaudio
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor

model_id = "sorika-labs/vaani1-lite-asr"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if torch.cuda.is_available() else torch.float32

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    torch_dtype=dtype,
    low_cpu_mem_usage=True
).to(device)

audio, sr = torchaudio.load("audio.wav")
if sr != 16000:
    audio = torchaudio.transforms.Resample(sr, 16000)(audio)

inputs = processor(audio.squeeze().numpy(), sampling_rate=16000, return_tensors="pt")
input_features = inputs.input_features.to(device, dtype=dtype)

with torch.no_grad():
    predicted_ids = model.generate(input_features, language="hi", task="transcribe")

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
print("Result:", transcription[0])

---

⚙️ Architecture & Model Specs

• Base Model: openai/whisper-small (244 Million Parameters)
• Audio Encoder: 12-layer Transformer, 768 hidden size, 80 mel channels
• Text Decoder: 12-layer Transformer, 768 hidden size
• Adapter Weights: 100% permanently merged into model.safetensors
• Optimal Audio Format: 16 kHz Mono WAV/FLAC

---

🏢 Developed by Sorika Labs

Sorika Labs is building high-efficiency, sovereign speech and multimodal foundation AI.
For technical support or inquiries, visit Sorika Labs on Hugging Face.