Qwen3-ASR-1.7B-Yue

A Cantonese (粵語) speech-recognition model, fine-tuned from Qwen3-ASR-1.7B. It writes Hong Kong Traditional Chinese in the written-Cantonese spelling used on jyutping.org and words.hk, handles Cantonese-English code-switching, and supports context ("hotword") biasing through the system prompt.

What is different from the base model

• Written Cantonese output. Writes 嘅 咗 啲 喺 冇 嗰 嘢 係 and the other standard Cantonese forms, in Traditional characters, instead of Mandarin-style or Simplified text.
• A consistent spelling standard. Training labels follow the jyutping.org conventions (for example 噉 (pronoun) vs 咁 (adverb) kept apart, 畀, 鍾意, 而家, 搵, 返 as the resumptive suffix) and sentence-final particles chosen by the tone that is heard.
• Code-switching. Cantonese with embedded English words is part of the training mixture.
• Rare and colloquial words. Synthetic clips cover rare characters and vulgar or colloquial words.
• Hotwords that work. The model was trained with contexts in the system prompt, including wrong or unrelated contexts, so it uses words it is given for spelling and recognition without writing words it did not hear.
• Nothing lost elsewhere. English and Mandarin replay data were kept in the mixture so those languages do not degrade.

Usage

pip install -U qwen-asr

import torch
from qwen_asr import Qwen3ASRModel

model = Qwen3ASRModel.from_pretrained(
    "hon9kon9ize/Qwen3-ASR-1.7B-Yue",
    dtype=torch.bfloat16,
    device_map="cuda",
    max_new_tokens=256,
)

result = model.transcribe(audio="audio.wav", language="Cantonese")
print(result[0].text)

language="Cantonese" forces Cantonese output; leave it out to let the model detect the language.

Hotwords / context

Pass names and terms through context. A Vocabulary: list or 詞彙： followed by the words works best for exact spelling:

result = model.transcribe(
    audio="audio.wav",
    language="Cantonese",
    context="詞彙：九龍塘、港鐵、恒生",
)

If the context contains words that are not in the audio they are not written. Contexts that closely match the next words of the audio can make the model repeat them, so keep contexts to short word lists.

Serving with vLLM

The model also loads in vLLM. For audio longer than about 158 s the default encoder cache rejects the request; serve with --max-num-batched-tokens 16384 --max-model-len 16384, or chunk the audio into pieces of about 40 s cut at quiet points.

Training

Starting point

1. Vocabulary extension. 231 new token rows were added for Cantonese characters that the base tokenizer splits into byte pieces (嘅, 咗, 啲, 喺, 冇, ...). Each new input and output embedding row is initialised from the mean of the rows of the pieces it replaces, so the extended model starts out equivalent to the original.
2. Continued pre-training of the decoder (language prior). The decoder language model (Qwen3-1.7B) was continued-pre-trained on a 515M-character Cantonese text corpus built from Cantonese YouTube videos and podcast audio. The resulting decoder delta was merged into the extended ASR model at a weight of 0.5, which gives the starting checkpoint.

Fine-tuning

LoRA (rank 32, alpha 32) on the decoder, plus training of the 231 new embedding rows.

The adapter was merged into the weights (the released model needs no adapter).

Context training. 60% of the clips get a context: the clip's own rare, English or content words as a list; a template sentence containing them; a different transcript that contains the word; deliberately wrong words (so the model learns not to write what it did not hear); or unrelated text. The other 40% have an empty prompt.

Homophone swap. For 40% of eligible clips, one character of a hotword is replaced by a same-sound character in both the label and the context. The label then depends on the context, which teaches the model to follow the spelling it is given.

Training data

29,396 clips, about 39 hours, after cleaning.

Label cleaning and standardisation

All Cantonese labels went through these steps:

• Hygiene. Rows with Simplified-only text were removed, look-alike typos were repaired (for example 出黎 to 出嚟, 既野 to 嘅嘢, 我地 to 我哋), and Common Voice rows with a censored ? were dropped.
• Spelling standard. Labels follow the jyutping.org list of common Cantonese typos: 噉 / 咁 kept as two words (an automatic decision model separated the pronoun and the adverb uses, with human verdicts on the uncertain cases), 俾 to 畀, 中意 / 鐘意 to 鍾意, 番 to 返 for the resumptive suffix, 依家 to 而家, 揾 to 搵, 僕街 to 仆街, and a set of look-alike code points.
• Sentence-final particles. About 1,300 particle characters were set from the syllable and tone that a Jyutping recogniser hears, and checked by ear on a sample (啊 aa3, 呀 aa4, 吖 aa1, 啦 laa1, 喇 laa3, 嘞 laak3, 囉 lo1, 咯 lok3).
• Alignment filter. Common Voice clips whose audio does not follow the text were removed (195 clips: those where both a Jyutping recogniser and a Qwen model disagreed with the label on at least 15% of syllables, plus clips judged bad when listened to).
• Human corrections from a listening pass were applied.

Intended use and limitations

• Intended use: transcribing spoken Cantonese, including code-switched speech, with optional vocabulary biasing.
• Orthography is a choice. The model writes one consistent standard. If your references use other conventions (for example 系 for 係, 咁 for 噉, 俾 for 畀, 翻 for 返, or Simplified Chinese) a character-level comparison will count those as errors; normalise spelling before comparing.
• Output script. Output is Traditional Chinese. Traditional characters can also appear in Mandarin output.
• Repetition loops. Rarely (a few outputs in thousands), a hallucinated repeating phrase appears; one code-switched clip produced a long runaway English paragraph. If the output is much longer than the audio would allow, runs above about 7 characters per second, or repeats a unit five or more times, decode again without context, or split long audio at pauses.
• Long audio. See the vLLM note above. Whole-clip decoding is usually better than short chunking.
• Rare names and classical vocabulary are the main error source. Supplying them in the context helps.
• Telephone speech. Telephone-channel augmentation was used in training, but real call recordings have not been measured systematically.
• Not for speaker identification, speech with strong non-Cantonese accents, or safety-critical use.

License and data

The weights are released under Apache-2.0, the same licence as the base model Qwen3-ASR-1.7B.

Citation

@misc{qwen3asr_yue,
  title  = {Qwen3-ASR-1.7B-Yue},
  author = {hon9kon9ize}
  year   = {2026},
  url    = {https://huggingface.co/hon9kon9ize/Qwen3-ASR-1.7B-Yue}
}

This model is built on Qwen3-ASR from the Qwen team. Please also cite their technical report:

@article{Qwen3-ASR,
  title={Qwen3-ASR Technical Report},
  author={Xian Shi, Xiong Wang, Zhifang Guo, Yongqi Wang, Pei Zhang, Xinyu Zhang, Zishan Guo, Hongkun Hao, Yu Xi, Baosong Yang, Jin Xu, Jingren Zhou, Junyang Lin},
  journal={arXiv preprint arXiv:2601.21337},
  year={2026}
}