manassehzw/sna-qwen-asr-1.7b
Shona Qwen3-ASR 1.7B is a Shona (sna) automatic speech recognition model
fine-tuned from Qwen/Qwen3-ASR-1.7B
on manassehzw/sna-dataset-annotated.
Overview
This release contains checkpoint 760 from run 20260803T140000Z. It was selected
by generated validation WER and occurs at approximately two epochs. Qwen3-ASR
adapted rapidly to Shona and substantially outperformed its zero-shot behavior on
the project dataset. Evaluation on FLEURS also revealed a meaningful domain and
speaker-generalization gap, so the in-domain score should not be treated as a
universal Shona ASR result.
Model Details
- Curated by: Manasseh Changachirere (Harare Institute of Technology)
- Base model:
Qwen/Qwen3-ASR-1.7B - Base model revision:
7278e1e70fe206f11671096ffdd38061171dd6e5 - Training dataset:
manassehzw/sna-dataset-annotated - Dataset revision:
f91b1a79cbac15520d3c808b56e5192bf903f280 - Training run:
20260803T140000Z - Released checkpoint:
checkpoint-760 - Checkpoint epoch: approximately
1.996 - Selection criterion: lowest generated WER among retained checkpoints
- Framework:
qwen-asr==0.0.6, Transformers, PyTorch
Evaluation
Text for the project evaluations was normalized with Unicode NFKC, case folding, punctuation and symbol removal, and whitespace collapse.
| Evaluation | Checkpoint | Examples | WER | CER |
|---|---|---|---|---|
| In-domain validation generation probe | 760 | 256 | 25.20% | 5.08% |
| Full in-domain test set | 855 | 1,565 | 25.51% | 4.97% |
PazaBench v2 / FLEURS sn_zw test |
855 | 925 | 52.85% | 13.57% |
| Zero-shot base model, full in-domain test set | base | 1,565 | 106.58% | 31.64% |
The full-test and FLEURS numbers are included as nearby-run context and were measured on checkpoint 855, not the released checkpoint. Checkpoints 760 and 855 were effectively tied on the 256-example generation probe: 25.198% versus 25.215% WER, a difference of one word error. A full external evaluation of checkpoint 760 has not yet been recorded.
Training Summary
- Training examples: 12,170
- Validation examples: 1,504
- Held-out test examples: 1,565
- Training audio: approximately 62.63 hours
- Learning rate:
2e-5 - Microbatch size: 4
- Gradient accumulation: 8
- Effective batch size: 32
- Precision: bfloat16
- Scheduler: linear with 2% warmup
- Planned epochs: 3
- Released checkpoint: approximately 2 epochs / 760 optimizer steps
- Checkpoint evaluation loss:
0.17747
Example Usage
Install the official Qwen ASR package:
pip install -U qwen-asr
import torch
from qwen_asr import Qwen3ASRModel
model = Qwen3ASRModel.from_pretrained(
"manassehzw/sna-qwen-asr-1.7b",
dtype=torch.bfloat16,
device_map="cuda:0",
max_inference_batch_size=16,
max_new_tokens=256,
)
results = model.transcribe(audio="sample.wav", language=None)
print(results[0].text)
The recorded evaluations used language=None. Although the fine-tuning targets
identify the output as Shona, Shona was not in the base model's original list of
supported language arguments, so automatic language handling is the tested path.
Limitations
- External FLEURS performance is substantially weaker than in-domain performance.
- The training corpus is relatively small and may not cover Zimbabwe's full range of speakers, accents, recording conditions, and conversational domains.
- Code-switching was not separately quantified for this checkpoint.
- Long-form, streaming, noisy, telephone, and far-field behavior require further evaluation.
- Orthographically close substitutions can produce a low CER while still being penalized heavily by WER.
License
The model is released under Apache-2.0, matching the base Qwen3-ASR model.