Whisper Small Sinhala v1

whisper-small-sinhala-v1 is a fine-tuned version of OpenAI Whisper Small for automatic speech recognition (ASR) in Sinhala. It is intended to convert Sinhala speech into written Sinhala text.

This repository is currently gated. Request access through the model page before downloading the weights.

Model details

Intended use

The model is suitable for research, prototyping, and Sinhala speech-transcription experiments. Potential applications include transcription tools, subtitle preparation, voice interfaces, and accessibility workflows.

Performance can vary with accent, background noise, recording quality, speaking rate, and domain-specific vocabulary. Review generated transcripts before using them in high-impact workflows.

Usage

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline

model_id = "kaan84/whisper-small-sinhala-v1"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    torch_dtype=torch_dtype,
    low_cpu_mem_usage=True,
    use_safetensors=True,
)
model.to(device)

processor = AutoProcessor.from_pretrained(model_id)

asr = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    torch_dtype=torch_dtype,
    device=device,
)

result = asr("sample.wav", generate_kwargs={"language": "si", "task": "transcribe"})
print(result["text"])

Training data

The repository metadata does not currently provide a complete dataset card. Add the following before treating this model as production-ready documentation:

• Dataset name, source, and version
• Number of audio hours and speakers
• Train, validation, and test split sizes
• Audio preprocessing and filtering steps
• Licensing and consent information

Training procedure

This model was fine-tuned from openai/whisper-small. Training hyperparameters, hardware, checkpoint-selection criteria, and preprocessing settings should be added here to make the result reproducible.

Evaluation

The model was evaluated during training on a fixed test subset of the Sinhala ASR dataset.

Evaluation setup

Results

Validation loss decreased from 0.7160 at step 100 to 0.1546 at step 2,000, indicating consistent improvement throughout training.

Qualitative example

Reference

මේ නිසා බොහෝ දෙනකු

Prediction

මේ නිසා බොහෝ දෙනෙකු

For this individual example, the word error rate was 25.0% and the character error rate was approximately 5.6%. The difference is a minor spelling variation.

Evaluation limitations

Corpus-level WER and CER were not calculated across the complete 10,000-sample evaluation subset. Therefore, the example-level error rates above should not be interpreted as the model’s overall performance.

The model remains research-grade. Performance may vary depending on speaker accent, recording quality, background noise, audio duration, and domain-specific vocabulary.

Limitations

• The model may produce incorrect or incomplete transcriptions.
• Accuracy may decrease for noisy, overlapping, distant, or low-quality speech.
• Coverage may be uneven across Sinhala accents, regions, ages, and specialized domains.
• The model may reproduce biases or errors present in its training data.
• Human verification is recommended for legal, medical, educational-assessment, or other consequential use.

License

Apache License 2.0. Confirm that all training datasets and redistributed assets are compatible with this license.

Citation

If you use this model, cite the original Whisper paper and link to this model repository. Add a project-specific BibTeX entry when the training work is formally published.