Whisper Medium — Malayalam R-MFT

This model was introduced in the paper Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition.

Fine-tuned Malayalam ASR model based on openai/whisper-medium, trained using the Reverse Multi-Stage Fine-Tuning (R-MFT) recipe introduced in Vividh-ASR: Diagnosing and Fixing Studio-Bias in Whisper for Indic Languages.

This model is part of a set of Malayalam and Hindi Whisper models released by Adalat AI alongside the Vividh-ASR benchmark.

---

Model Description

R-MFT trains in three stages with a decreasing learning rate schedule, presenting the hardest acoustic data first during the highest-plasticity phase:

Training uses AdamW (weight decay 0.1), linear warmup for the first 10% of steps, and cosine annealing to zero. Trained on NVIDIA H100 GPUs using HuggingFace Transformers.

---

Benchmark Results (Vividh-ASR)

Benchmark WER is measured using faster-whisper with 7s VAD segmentation for long-form audio. See the blogpost for full evaluation details.

WER %. Lower is better. See Vividh-ASR benchmark for full evaluation details.

---

Usage

from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="adalat-ai/whisper-medium-ml-rmft",
    chunk_length_s=30,
    device="cuda"
)

result = asr("audio.wav")
print(result["text"])

Note: For long-form audio, benchmark results use faster-whisper with 7s VAD segmentation. For short clips, the HuggingFace pipeline above will produce equivalent results.

---

Training Data

Training data is a superset of the Vividh-ASR benchmark evaluation splits. Sources used:

---

Intended Use & Limitations

This model is intended as a general-purpose Malayalam ASR model optimised for verbatim transcription accuracy across diverse acoustic conditions.

Limitations:

• Evaluated on Hindi and Malayalam only; generalisation to other Indic languages is untested
• Tier D evaluation uses synthetic noise profiles; performance on real-world degraded audio may differ

---

Citation

If you use this model or the Vividh-ASR benchmark, please cite:

@misc{vividhasr2025,
  title   = {Vividh-ASR: Diagnosing and Fixing Studio-Bias in Whisper
             for Indic Languages},
  author  = {Kush Juvekar, Kavya Manohar, Kumaramanas Nethil},
  year    = {2026},
  url     = {https://huggingface.co/blog/adalat-ai/vividh-benchmark}
}

@misc{vividh2026,
      title={Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition}, 
      author={Kush Juvekar, Kavya Manohar, Aditya Srinivas Menon, Arghya Bhattacharya, Kumarmanas Nethil},
      year={2026},
      eprint={2605.13087},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2605.13087}, 
}

---

Related Models and Datasets

See the Vividh collection.

---

Developed by Adalat AI. Released under Apache 2.0.