Qwen3-ASR-1.7B fine-tuned on ViMD (full fine-tune)

All 2.04B parameters trained (audio encoder + LLM), bf16 weights, lr 1e-5, 8-bit paged AdamW. Training took 126 min (peak 18.1 GB). This is a complete model, not an adapter.

Usage

import torch
from qwen_asr import Qwen3ASRModel   # pip install qwen-asr

w = Qwen3ASRModel.from_pretrained("duclvQ/qwen3-asr-1.7b-vimd-full", dtype=torch.bfloat16, device_map="cuda:0")
print(w.transcribe(audio="audio.wav", language="Vietnamese")[0].text)

Results

Held-out test split of ViMD (Vietnamese dialect speech, North + Central regions; 1,368 clips, 7.2 h, 939 speakers, none seen in training). WER/CER are computed after lowercasing and stripping punctuation, with language="Vietnamese" forced.

Training

• Base: Qwen/Qwen3-ASR-1.7B. Data: ViMD North + Central train split, 10,346 clips (~55 h), clips <= 30 s. Transcripts are verbatim: numbers spelled out, fillers and repetitions kept.
• Target format: language Vietnamese<asr_text>{transcript}. 2 epochs (1,294 steps), effective batch 16, cosine schedule, bf16, gradient checkpointing, seed 42.

Quantized variants (subfolders q8/ and q4/)

Weight-only HQQ quantization (no calibration data) of the LLM linear layers; the audio encoder stays bf16. Same test protocol as above, 400 random test clips (bf16 on the same clips: 8.68 WER).

6-bit (round-to-nearest simulation) lost nothing but HQQ has no 6-bit packing, so it is not published. 3-bit and 2-bit collapse with this method (WER 100%+ at q3; q2 hallucinates in all group sizes tried), so they are not published either. Calibrated methods (AWQ/GPTQ) or QAT would be needed for those.

HQQ inference in transformers is slow (it dequantizes each matmul): ~2x slower than bf16 here. These checkpoints are for memory savings and experimentation, not throughput.

# pip install qwen-asr hqq ; load_hqq_asr.py is in this repo
from huggingface_hub import snapshot_download
from load_hqq_asr import load_hqq_asr

path = snapshot_download("duclvQ/qwen3-asr-1.7b-vimd-full", allow_patterns=["q4/*"])   # or "q8/*"
w = load_hqq_asr(path + "/q4")
print(w.transcribe(audio="audio.wav", language="Vietnamese")[0].text)

Outputs begin with language Vietnamese<asr_text>; strip it (as for the bf16 model).

Caveats

• One seed and one hyperparameter setting per method; differences below ~0.5 WER points are not reliable.
• Part of the improvement over the base model is format learning: the base model writes digits (e.g. "2022") where the references spell numbers out, and handles fillers differently. Excluding digit clips and fillers, the neutralised gain for LoRA is 8.51 -> 6.69 WER (about 64% of the raw gain).
• Evaluated on ViMD only (read/spontaneous dialect speech); no claims about other domains. Check the ViMD dataset license before any commercial use of this model.
• Base model license: Apache-2.0 (Qwen3-ASR).