CT2 Whisper Medium — Malayalam R-MFT

CTranslate2 conversion of adalat-ai/whisper-medium-ml-rmft, optimised for fast CPU/GPU inference via faster-whisper.

This model was introduced in the paper Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition.

---

Model Description

The source model is a fine-tuned Malayalam ASR model based on openai/whisper-medium, trained using the Reverse Multi-Stage Fine-Tuning (R-MFT) recipe introduced in Vividh-ASR: Diagnosing and Fixing Studio-Bias in Whisper for Indic Languages.

R-MFT trains in three stages with a decreasing learning rate schedule, presenting the hardest acoustic data first during the highest-plasticity phase:

---

Benchmark Results (Vividh-ASR)

Benchmark WER is measured using faster-whisper with 7s VAD segmentation for long-form audio. See the blogpost for full evaluation details.

WER %. Lower is better.

---

Usage

from faster_whisper import WhisperModel

model = WhisperModel(
    "adalat-ai/ct2-whisper-medium-ml-rmft",
    device="cuda",
    compute_type="float16"
)
segments, info = model.transcribe("audio.wav", vad_filter=True, vad_parameters={"max_speech_duration_s": 7})

for segment in segments:
    print(f"{segment.start:.2f} - {segment.end:.2f}: {segment.text}")

Note: Benchmark results use 7s VAD segmentation (vad_filter=True, max_speech_duration_s=7). For short clips, VAD is not required.

---

Training Data

Training data is a superset of the Vividh-ASR benchmark evaluation splits.

---

Citation

If you use this model or the Vividh-ASR benchmark, please cite:

@misc{vividhasr2025,
  title   = {Vividh-ASR: Diagnosing and Fixing Studio-Bias in Whisper
             for Indic Languages},
  author  = {Kush Juvekar, Kavya Manohar, Kumaramanas Nethil},
  year    = {2026},
  url     = {https://huggingface.co/blog/adalat-ai/vividh-benchmark}
}

@misc{vividh2026,
      title={Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition}, 
      author={Kush Juvekar, Kavya Manohar, Aditya Srinivas Menon, Arghya Bhattacharya, Kumarmanas Nethil},
      year={2026},
      eprint={2605.13087},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2605.13087}, 
}

---

Related Models and Datasets

• Source Transformers model: adalat-ai/whisper-medium-ml-rmft
• See the full Vividh collection.

---

Developed by Adalat AI. Released under Apache 2.0.