The model card, file list, and checksums are here. Have the files and permission to share them? Submit a torrent so others can download from peers.
Getting the source files requires Hugging Face approval.This model is a fine-tuned version of the Whisper v3 model, specifically trained for automatic speech recognition (ASR) in Cantonese (Yue). The model has been fine-tuned on data from the Common Voice 17 dataset for 10 epochs with a learning rate of 1e-7.
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
This model is intended for researchers and developers interested in building applications that require speech recognition capabilities in Cantonese. It can be used in various applications, including:
The model is specifically fine-tuned for Cantonese and may not perform well on other languages or dialects. Performance may vary based on the quality and accent of the audio input. The model's effectiveness is dependent on the diversity and richness of the training data.
To use this model, you can load it using the Hugging Face Transformers library:
from transformers import WhisperProcessor, WhisperForConditionalGeneration
model = WhisperForConditionalGeneration.from_pretrained("your_username/whisper-cantonese")
processor = WhisperProcessor.from_pretrained("your_username/whisper-cantonese")
Common Voice_17_0 yue test split Common Voice 15_0 yue test split and Common Voice 15_0 zh-HK test split (these test dataset were used to evaluate Whisper 3.0)
Character Error Rate (CER) since Cantonese is character based language.
| CV15_0 zh-HK | CV 15_0 yue | CV 17_0 yue | |
|---|---|---|---|
| Whisper large v3 | 10.8 | 16 | - |
| Whisper cantonese (ours) | 18.88 | 8.77 | 7.26 |
Explanation: our model was not trained on zh-HK data consisting of more written Cantonese but rather more vernacular Cantonese version (yue) since it is a speech recognition model. Hence the weaker performance on zh-HK splits of Common Voice dataset.
BibTeX:
@misc {rita_frieske_2025, author = { {Rita Frieske} }, title = { whisper-large-v3-cantonese }, year = 2025, url = { https://huggingface.co/khleeloo/whisper-large-v3-cantonese }, doi = { 10.57967/hf/4393 }, publisher = { Hugging Face } }
model-00001-of-00002.safetensors 5.0 GBmodel-00002-of-00002.safetensors 1.2 GBtraining_args.bin 5.6 kB