Qwen3-ASR-1.7B fine-tuned on ViMD (QLoRA)
LoRA adapter trained on a 4-bit NF4 (double-quant) base LLM, audio encoder and lm_head kept in bf16; same LoRA config as the LoRA model (17.4M trainable). lr 1e-4, 8-bit paged AdamW. Training took 75 min (peak 13.3 GB). The numbers above are for the adapter merged into the bf16 base, not the 4-bit base it was trained against.
Usage
import torch
from peft import PeftModel
from qwen_asr import Qwen3ASRModel # pip install qwen-asr
w = Qwen3ASRModel.from_pretrained("Qwen/Qwen3-ASR-1.7B", dtype=torch.bfloat16, device_map="cuda:0")
w.model = PeftModel.from_pretrained(w.model, "duclvQ/qwen3-asr-1.7b-vimd-qlora").merge_and_unload().eval()
print(w.transcribe(audio="audio.wav", language="Vietnamese")[0].text)
Results
Held-out test split of ViMD (Vietnamese dialect speech, North + Central regions; 1,368 clips, 7.2 h, 939 speakers, none seen in training). WER/CER are computed after lowercasing and stripping punctuation, with language="Vietnamese" forced.
| System | WER (all) | CER (all) | WER North | WER Central |
|---|---|---|---|---|
| Qwen3-ASR-1.7B (zero-shot) | 11.56 | 8.12 | 8.56 | 15.45 |
| LoRA | 8.72 | 6.29 | 6.59 | 11.47 |
| QLoRA | 9.45 | 6.8 | 7.23 | 12.31 |
| Full fine-tune | 8.59 | 6.17 | 6.6 | 11.17 |
Training
- Base:
Qwen/Qwen3-ASR-1.7B. Data: ViMD North + Central train split, 10,346 clips (~55 h), clips <= 30 s. Transcripts are verbatim: numbers spelled out, fillers and repetitions kept. - Target format:
language Vietnamese<asr_text>{transcript}. 2 epochs (1,294 steps), effective batch 16, cosine schedule, bf16, gradient checkpointing, seed 42.
Caveats
- One seed and one hyperparameter setting per method; differences below ~0.5 WER points are not reliable.
- Part of the improvement over the base model is format learning: the base model writes digits (e.g. "2022") where the references spell numbers out, and handles fillers differently. Excluding digit clips and fillers, the neutralised gain for LoRA is 8.51 -> 6.69 WER (about 64% of the raw gain).
- Evaluated on ViMD only (read/spontaneous dialect speech); no claims about other domains. Check the ViMD dataset license before any commercial use of this model.
- Base model license: Apache-2.0 (Qwen3-ASR).