omniASR CTC 300M_v2 — fine-tuned on Chichewa/Nyanja (synthetic only)
Fine-tuned checkpoint of OmniLingual ASR CTC 300M_v2
on the CLEAR Global Chichewa Synthetic ASR Dataset (374,576 examples, ~550 hours of TTS-synthesised speech).
Training used the fairseq2 wav2vec2 ASR recipe for 5,000 steps with the encoder frozen for the first
1,000 steps. The FLEURS Chichewa dev split was used for validation (the synthetic dataset has no dev split).
Tokenizer: omniASR_tokenizer_written_v2.
Results
| Dataset | WER (%) | CER (%) |
|---|---|---|
| FLEURS Chichewa test | 58.70 | 14.92 |
| Zambezi Voice Nyanja test | 70.47 | 16.56 |
For reference, the zero-shot omniASR CTC 300M_v2 baseline reaches 51.92% WER on FLEURS and 71.87%
on Zambezi Voice. Synthetic-only fine-tuning is worse than zero-shot on FLEURS due to acoustic
domain mismatch between TTS and natural speech, while providing a small improvement on the
out-of-domain Zambezi Voice benchmark. See also the FLEURS fine-tuned variant
(ai4good-labyrinth/omniASR-CTC-300M-v2-chichewa) which achieves 36.53% on FLEURS.
Training details
- Base model: omniASR_CTC_300M_v2
- Tokenizer: omniASR_tokenizer_written_v2
- Training data: CLEAR-Global/Chichewa-Synthetic-ASR-Dataset train split (374,576 examples, ~550 h)
- Steps: 5,000 (best checkpoint at step 2,500 by FLEURS dev WER)
- Dev WER at best checkpoint: 59.12% (FLEURS dev)
- Learning rate: 1e-5
- Encoder frozen: first 1,000 steps
- Gradient accumulation: 4 batches
- Mixed precision: bfloat16
- Framework: fairseq2
Checkpoint format
The checkpoint is stored in fairseq2's sharded format.
checkpoint/model/pp_00/tp_00/sdp_00.pt contains the full model state
(training used a single GPU, so there is only one shard).
To load the model, use the OmniLingual ASR inference pipeline with the
omniASR_CTC_300M_v2 model card and point the checkpoint path at sdp_00.pt.