Qwen3-ASR-1.7B-Cantonese-HK

Full-parameter fine-tune of Qwen/Qwen3-ASR-1.7B (revision 7278e1e) for Hong Kong Colloquial Cantonese (yue-HK) + English code-switching, with context / hotword vocabulary biasing support. It natively outputs Hong Kong Traditional Chinese with colloquial characters (係, 喺, 嘅, 咗, 㗎, 閂) and keeps English words in English.

This repo holds the original BF16 weights in Hugging Face Transformers format (config.json + model.safetensors). It is a full fine-tune, so there is no separate LoRA / PEFT adapter.

Current version: v2.1 (2026-11-14) — adds context / domain-vocabulary biasing alignment (reducing short-utterance + long-glossary CER from 21.88% to 7.19% with 0.0% prompt parroting) and improves 5 of 6 standard Cantonese benchmarks over v2 (v2 is revision 5c1df898, v1 is revision 399c5ef). GitHub: joe2643/qwen3-asr-cantonese-hk (quick start, client and evaluation scripts).

Other formats

Repo	Format	For
this repo	BF16, Transformers (v2.1)	`transformers`, `qwen-asr`, vLLM, further fine-tuning, conversion
this repo, `comfyui/`	BF16, ComfyUI-Drama3 layout (v2.1)	ComfyUI-Drama3 native Qwen3-ASR loader
`-GGUF`	GGUF Q4_K_M / Q6_K / Q8_0 + mmproj (v2.1)	`llama.cpp` (`llama-server`), OraType
`-FP8`	FP8 W8A8 (compressed-tensors, v2)	vLLM, more KV-cache room
`-NVFP4`	NVFP4A16 (compressed-tensors, v2)	vLLM, smallest and fastest

Benchmark (Character Error Rate, greedy, BF16 `transformers`)

Test Set	Stock `Qwen3-ASR-1.7B`	v1	v2	v2.1 (this)	`whisper-large-v3`
FLEURS (`yue_hant_hk`)	4.20%	4.00%	3.95%	3.89%	20.97%
Common Voice `yue`	5.46%	3.80%	3.86%	3.94%	36.44%
Common Voice 23 `zh-HK`	7.65%	5.18%	5.09%	5.02%	41.76%
MDCC	4.25%	2.53%	2.49%	2.47%	34.52%
LegCo (HK Legislative Council)	7.22%	7.17%	7.16%	7.16%	32.15%
MCE (Cantonese-English Code-Switching)	11.74%	5.61%	5.62%	5.52%	24.00%

Context / Hotword Vocabulary Biasing Benchmark (210-clip suite)

Condition (70 clips each)	v2 CER	v2.1 CER	v2.1 Prompt Parroting
Short command + long glossary (`short_long_prompt`)	21.88%	7.19%	0.0%
Homophone / domain target hit (`hit_homo`)	7.50%	6.54%	0.0%
Pure distractor glossary (`pure_distractor`)	10.14%	10.02%	0.0%

Before scoring, text is converted to Simplified (OpenCC t2s), numbers are normalised and spaces removed. Language is forced to Cantonese. Only Cantonese and Cantonese–English speech were evaluated; other languages may be worse than the stock model.

Usage (`qwen-asr`, Transformers backend)

import torch
from qwen_asr import Qwen3ASRModel  # pip install qwen-asr

asr = Qwen3ASRModel.from_pretrained("joe2643/Qwen3-ASR-1.7B-Cantonese-HK",
                                    dtype=torch.bfloat16, device_map="cuda:0", max_new_tokens=256)
print(asr.transcribe(audio="sample.wav", language="Cantonese")[0].text)

# Optional hotword / domain vocabulary biasing (placed in system prompt under the hood)
print(asr.transcribe(
    audio="command.wav",
    language="Cantonese",
    context="閂燈, 開燈, 閂冷氣, OraType, GitHub, PR",
)[0].text)

ComfyUI (ComfyUI-Drama3)

comfyui/qwen3_asr_1.7b_cantonese_hk_bf16.safetensors was made with ComfyUI-Drama3's own converter (commit 8cb3c64):

hf download joe2643/Qwen3-ASR-1.7B-Cantonese-HK comfyui/qwen3_asr_1.7b_cantonese_hk_bf16.safetensors --local-dir ComfyUI/models/asr

Put it in ComfyUI/models/asr/ and pick it in Load Qwen3-ASR; set language to Cantonese in Qwen3-ASR Transcribe.

Used by

OraType: free, open-source voice typing for macOS. It ships the GGUF Q6_K with the mmproj-Q8_0 encoder as its built-in speech model for Cantonese.

Support

If this model is useful to you, consider buying me a lemon tea (請飲杯檸檬茶) to support future open-source Cantonese speech training runs: buymeacoffee.com/joe2643 🍋

Credits

Fine-tuning, quantization, and GGUF export: @joe2643 (HF)
Fine-tuning strategy, data collection direction: @sennett-lau
ComfyUI conversion tool: ComfyUI-Drama3 by PedroMarinhoDev

joe2643/Qwen3-ASR-1.7B-Cantonese-HK