Whisper-small-vaani-kannada

This is a fine-tuned version of OpenAI's Whisper-Small, trained on Kannada speech from multiple datasets.

Usage

This can be used with the pipeline function from the Transformers module.

import torch
from transformers import pipeline

audio = "path to the audio file to be transcribed"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
modelTags="ARTPARK-IISc/whisper-small-vaani-kannada"
transcribe = pipeline(task="automatic-speech-recognition", model=modelTags, chunk_length_s=30, device=device)
transcribe.model.config.forced_decoder_ids = transcribe.tokenizer.get_decoder_prompt_ids(language="ka", task="transcribe")

print('Transcription: ', transcribe(audio)["text"])

Training and Evaluation

The models has finetuned using folllowing dataset Vaani , Fleurs,IndicTTS

The performance of the model was evaluated using multiple datasets, and the evaluation results are provided below.