Qwen3-ASR-1.7B Malay LoRA (lora-1.7b-malay50h-babel)

LoRA adapter (r=16, alpha=32, decoder attention + MLP, 17.4 M trainable parameters) that fine-tunes Qwen/Qwen3-ASR-1.7B for Malay and Malay-English code-switched speech. The merged, ready-to-serve checkpoint is at duclvQ/qwen3-asr-1.7b-malay-merged. Code, data pipeline, full report and serving stack: github.com/duclvq/finetune_QwenASR.

Training data (83 h)

Speaker spk001 of BabelSpeech (377 clips, 1.8 h) was held out entirely as the real code-switch test. 2 epochs, lr 1e-4 cosine, effective batch 16, bf16, seed 42. RTX 5060 Ti 16 GB + a Colab A100 resume.

Results (WER %, lower is better)

A Malay-only LoRA on the same 50 h scored 16.14 on Revolab (worse than base: it rewrote English words as Malay); the code-switch and English data fix that. About a third of the gain on the real code-switch set is label-convention matching (e.g. "korang" vs "kau orang"); the rest is recognition. One seed per run; one real test speaker.

Usage

import torch
from peft import PeftModel
from qwen_asr import Qwen3ASRModel

w = Qwen3ASRModel.from_pretrained("Qwen/Qwen3-ASR-1.7B", dtype=torch.bfloat16, device_map={"": 0},
                                  max_inference_batch_size=8, max_new_tokens=400)
w.model = PeftModel.from_pretrained(w.model, "duclvQ/qwen3-asr-1.7b-malay-lora").merge_and_unload()
print(w.transcribe(audio=["clip.wav"], language="Malay")[0].text)

Outputs start with language Malay<asr_text>; strip that prefix. Cap generation (400 tokens for <= 30 s clips): on a few clips the decoder can loop, and an uncapped server will emit hundreds of repeated words.

Merge the adapter into the base and save a standalone checkpoint

Use this when you want a plain model directory for vLLM, HQQ or any loader that does not know PEFT. It is what produced duclvQ/qwen3-asr-1.7b-malay-merged.

import torch
from peft import PeftModel
from qwen_asr import Qwen3ASRModel

w = Qwen3ASRModel.from_pretrained("Qwen/Qwen3-ASR-1.7B", dtype=torch.bfloat16, device_map={"": 0})
w.model = PeftModel.from_pretrained(w.model, "duclvQ/qwen3-asr-1.7b-malay-lora").merge_and_unload()

# the base ships temperature=1e-6 with do_sample=False, which save_pretrained rejects
gc = w.model.generation_config
if gc is not None and not getattr(gc, "do_sample", False):
    gc.temperature = None

w.model.save_pretrained("qwen3-asr-1.7b-malay-merged", safe_serialization=True)
w.processor.save_pretrained("qwen3-asr-1.7b-malay-merged")

The output directory (~3.9 GB: model.safetensors, config.json, tokenizer and preprocessor files) loads with Qwen3ASRModel.from_pretrained("qwen3-asr-1.7b-malay-merged") or qwen-asr-serve qwen3-asr-1.7b-malay-merged. The repository's eval_quant.py --adapter <dir> does the same merge as its first step. Merging is lossless: the merged checkpoint scores the same as base + adapter (babel_test 8.62 WER both ways).

Tested with qwen-asr 0.0.6, transformers 4.57.6, peft 0.18.