This repository provides the standalone merged checkpoint (PL-HFT/qwen3-asr-0.6b-ko-dysarthria) created by merging PL-HFT/qwen3-asr-0.6b-ko-dysarthria-lora into Qwen/Qwen3-ASR-0.6B-hf in float32 and saving in bfloat16 (model.safetensors). Removing PEFT runtime wrapper hooks significantly speeds up startup and streaming inference.
When Qwen3-ASR-0.6B is fine-tuned solely on dysarthric speech, its error rate on typical Korean speech degrades sharply (4.63% -> 15.01% CER in full FT, 4.63% -> 8.02% in dysarthria-only LoRA). Joint 1-epoch LoRA fine-tuning on the combined corpus eliminates this cross-domain trade-off—reducing CER on both dysarthric (4.26% -> 0.79%) and typical (4.63% -> 1.97%) speech simultaneously.
Qwen/Qwen3-ASR-0.6B-hf (Qwen3-0.6B LLM + AuT audio encoder)PL-HFT/qwen3-asr-0.6b-ko-dysarthria-lora (r=64, lora_alpha=128, 250 modules)transformers >= 5.13.0 (Qwen3ASRForConditionalGeneration), no peft requiredmodel.safetensors (bfloat16, 611 tensors, 1,564,928,088 bytes)language Korean<asr_text>{transcript}<= 29.5sImportant:
Qwen3-ASR-0.6B-hfrequirestransformers >= 5.13.0(withQwen3ASRForConditionalGeneration), and outputs prefixed text likelanguage Korean<asr_text>...which should be stripped after<asr_text>.
import re
import torch
import torchaudio
from transformers import AutoProcessor, Qwen3ASRForConditionalGeneration
def extract_transcript(raw_output: str) -> str:
if "<asr_text>" in raw_output:
return raw_output.split("<asr_text>", 1)[1].strip()
return re.sub(r"^language\s+(?:<[^>]+>|[A-Za-z0-9_-]+)", "", raw_output).strip()
model_id = "PL-HFT/qwen3-asr-0.6b-ko-dysarthria" # or local folder path
device = "cuda" if torch.cuda.is_available() else ("mps" if torch.backends.mps.is_available() else "cpu")
dtype = torch.bfloat16 if device in ("cuda", "mps") else torch.float32
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3ASRForConditionalGeneration.from_pretrained(model_id, dtype=dtype)
model.eval().to(device)
# Load 16 kHz mono audio (<= 29.5s)
waveform, sr = torchaudio.load("sample.wav")
if sr != 16000:
waveform = torchaudio.functional.resample(waveform, sr, 16000)
if waveform.shape[0] > 1:
waveform = waveform.mean(dim=0, keepdim=True)
audio_np = waveform.squeeze(0).numpy()
inputs = processor.apply_transcription_request(audio=audio_np, language="Korean")
prepared = {
k: (v.to(device=device, dtype=dtype) if isinstance(v, torch.Tensor) and v.is_floating_point()
else (v.to(device=device) if isinstance(v, torch.Tensor) else v))
for k, v in inputs.items()
}
with torch.inference_mode():
output_ids = model.generate(**prepared, max_new_tokens=128, do_sample=False)
prompt_len = prepared["input_ids"].shape[1]
decoded = processor.batch_decode(output_ids[:, prompt_len:], skip_special_tokens=True)[0]
print(extract_transcript(decoded))
combine Dataset)The combine corpus merges two quality-verified Korean speech datasets preprocessed by Speech-dataset-pipeline (raw audio and transcripts are not redistributed in this repository):
dysarthria, AI-Hub 013):40.66% by segment count, 44.20% by duration).kresnik/wav2vec2-large-xlsr-korean), retaining strictly aligned Green segments (<= 29.5s, natural intra-utterance pauses preserved).57.9% (164.44h), Speech & Hearing 22.3% (63.36h), Neurological 19.7% (56.05h).typical, AI-Hub 109):59.34% by segment count, 55.80% by duration).| Split | Total Segments | Total Hours | Dysarthria (Segs / Hours / Speakers) | Typical (Segs / Hours / Speakers) |
|---|---|---|---|---|
| Train | 321,135 | 514.37 h | 129,885 / 227.72 h / 617 | 191,250 / 286.65 h / 473 |
| Val | 38,459 | 64.77 h | 16,531 / 28.93 h / 74 | 21,928 / 35.83 h / 55 |
| Test | 38,599 | 63.04 h | 15,495 / 27.21 h / 75 | 23,104 / 35.83 h / 55 |
train, val, and test.cross_split_overlap_count: 55,497). Evaluation separately reports Overlapped vs. Unseen prompt subsets.NVIDIA A100-SXM4-80GB (79.3 GB VRAM)4h 36m 05s (1.0 epoch, 5,020 steps)per_device_train_batch_size=16, gradient_accumulation_steps=4)adamw_torch_fused, learning_rate=1e-4, warmup_steps=350, weight_decay=0.01, bf16=true, tf32=true, seed=42eval_steps=400, save_steps=400, load_best_model_at_end=true (metric_for_best_model=loss), Final EvalLoss=0.0392transcript_normalized formatted as language Korean<asr_text>{text} with assistant-only loss maskingPeftModel.from_pretrained(...).merge_and_unload(safe_merge=True) in float32, saved as bfloat16 (model.safetensors SHA-256: 97b4770c35388ed676c798a1770f418c432d4e68626694028439edaf29b43ddf, max diff 0.0 vs. E2E-Application-CLI/caches/asr_merged)metadata/training_config.yaml and metadata/export_manifest.json.All Korean evaluations use the identical single-source normalizer (shared_controller.normalize_ko) and Levenshtein CER/WER scorer with 1,000-resample Bootstrap 95% confidence intervals. Benchmark numbers below were measured on the combine LoRA weights prior to merge; generation equivalence between LoRA and merged checkpoints was verified on reload.
N = 38,599)| Condition | Overall CER (95% CI) | Overall WER | Overlapped Prompts (N=34,105) CER |
Unseen Prompts (N=4,494) CER |
|---|---|---|---|---|
Base (Qwen/Qwen3-ASR-0.6B-hf) |
4.48% [0.0442 ~ 0.0455] |
16.00% | 4.27% | 5.61% |
| Combine LoRA / Merged (This Model) | 1.51% [0.0146 ~ 0.0156] |
5.70% | 1.02% | 4.08% |
| Evaluation Set | N | Base CER / WER | Combine CER / WER | Notes |
|---|---|---|---|---|
| Dysarthria Test (All) | 15,495 | 4.26% / 14.36% | 0.79% / 1.98% | -81.4% relative CER reduction |
↳ Laryngeal (후두장애) |
11,931 | 3.45% / — | 0.37% / — | |
↳ Speech & Hearing (언어청각장애) |
1,269 | 10.38% / — | 3.99% / — | |
↳ Neurological (뇌신경장애) |
2,295 | 4.84% / — | 1.09% / — | |
| Typical Test | 23,104 | 4.63% / 17.02% | 1.97% / 8.02% | -57.4% relative CER reduction |
External Korean Lecture (external_aihub71627) |
10,000 | 12.23% / 29.41% | 13.11% / 32.03% | Slight OOD drift vs. base |
E2E-Application), it is paired with a downstream medical LLM (google/medgemma-1.5-4b-it) for conservative error correction.Base, Combine achieves strong gains on both internal Korean dysarthria (0.79%) and typical (1.97%) tests, while showing mild regression on external Korean lecture audio (12.23% -> 13.11% CER) and cross-lingual English TORGO (38.13% CER / 58.32% WER under a separate protocol, reported as descriptive reference only) due to Korean-focused adaptation.1.02% CER) than on unseen prompts (4.08% CER).Qwen/Qwen3-ASR-0.6B-hf; training data derives from AI-Hub 013 & AI-Hub 109 and users must comply with applicable AI-Hub usage terms.PL-HFT/qwen3-asr-0.6b-ko-dysarthria-lora