Haitian Creole speech recognition, the second version of
Oswald-ASR-0.6-V1: fine-tuned from
Qwen/Qwen3-ASR-0.6B-hf, which does not speak Kreyòl.
Gold-transcript test sets: WER 9.1 / CER 3.5 (V1: 10.3 / 3.6), on 1316 clips, identical for both models. Numbers recognised whole: 90.3% (V1: 70.7%) of 290 numbers. Real phones, new speakers (Datamio, 10 speakers never trained on): WER 5.2 (V1: 11.5).
kreyol-fatiman-vocab, sentences full of dates, prices, times, phone numbers,
percentages, ages, names, places, stations and government services.scripts/whisper_ft/itn.py): amounts, dates, years, times, phone numbers, percentages, decimals and
units become digits and symbols. Scoring reads both models' output back into one canonical spoken form, so V2's
digits and V1's words are compared on the same words; a number counts as right only if it is right whole.V1 and V2 transcribed the same test clips, beam 4. Numbers is the share of the reference's numbers that come out whole: a year heard as "de mil kat" instead of "de mil vennkat" is wrong.
| Test set | Clips | V1 WER | V2 WER | V1 CER | V2 CER | V1 numbers | V2 numbers |
|---|---|---|---|---|---|---|---|
bible_read |
150 | 4.0 | 4.4 | 1.5 | 1.6 | 100% of 10 | 100% of 10 |
datamio_phone |
441 | 11.5 | 5.2 | 3.6 | 1.8 | 64% of 179 | 96% of 179 |
read_cmu |
150 | 6.8 | 7.1 | 2.2 | 2.3 | 78% of 27 | 81% of 27 |
studio_finance |
63 | 4.1 | 4.3 | 0.8 | 1.0 | 100% of 1 | 100% of 1 |
studio_health |
116 | 5.7 | 5.7 | 2.2 | 2.2 | 0% of 1 | 100% of 1 |
studio_history |
52 | 16.8 | 16.4 | 5.0 | 5.0 | 50% of 18 | 50% of 18 |
studio_legal |
46 | 3.2 | 3.9 | 0.8 | 1.1 | – | – |
studio_numbers |
85 | 6.7 | 5.0 | 1.8 | 1.6 | 100% of 39 | 95% of 39 |
studio_social |
93 | 7.0 | 6.9 | 2.3 | 2.2 | 100% of 4 | 100% of 4 |
youtube |
120 | 24.6 | 29.8 | 10.1 | 14.3 | 64% of 11 | 55% of 11 |
| Test set | Clips | V1 WER | V2 WER | V1 CER | V2 CER | V1 numbers | V2 numbers |
|---|---|---|---|---|---|---|---|
radio |
150 | 12.0 | 12.1 | 4.1 | 4.1 | 93% of 30 | 93% of 30 |
synth_vocab |
32 | 18.2 | 11.8 | 5.2 | 3.2 | 94% of 17 | 100% of 17 |
| Domain | Hours | Train / val / test clips | Gold |
|---|---|---|---|
radio |
51.0 | 16,340 / 1,276 / 1,918 | no |
bible_read |
26.8 | 9,966 / 223 / 493 | yes |
synth |
21.4 | 10,985 / 0 / 0 | no |
read_cmu |
12.1 | 3,782 / 344 / 360 | yes |
datamio_phone |
9.4 | 3,313 / 225 / 441 | yes |
synth_openai |
7.9 | 4,639 / 0 / 0 | no |
youtube |
7.0 | 2,396 / 114 / 120 | yes |
studio_health |
5.3 | 2,854 / 114 / 116 | yes |
bible_frht |
5.1 | 1,827 / 0 / 0 | yes |
bible_narrowband |
3.8 | 2,786 / 0 / 0 | yes |
studio_social |
3.7 | 2,116 / 96 / 93 | yes |
synth_vocab |
3.7 | 1,531 / 37 / 32 | no |
bible_frht_yt |
2.1 | 763 / 0 / 0 | yes |
studio_legal |
2.0 | 828 / 48 / 46 | yes |
studio_numbers |
1.4 | 1,008 / 83 / 85 | yes |
studio_finance |
1.2 | 669 / 66 / 63 | yes |
studio_history |
1.2 | 492 / 46 / 52 | yes |
synth_google |
0.7 | 553 / 0 / 0 | no |
synth_modern |
0.5 | 217 / 0 / 0 | no |
studio_ecom |
0.5 | 292 / 0 / 0 | yes |
from transformers import AutoProcessor, Qwen3ASRForConditionalGeneration
processor = AutoProcessor.from_pretrained("jsbeaudry/Oswald-ASR-0.6-V2")
model = Qwen3ASRForConditionalGeneration.from_pretrained("jsbeaudry/Oswald-ASR-0.6-V2", device_map="auto")
conversation = [[
{"role": "user", "content": [{"type": "audio", "path": "clip.wav"}]},
{"role": "assistant", "content": [{"type": "text", "text": "language Haitian Creole<asr_text>"}]},
]]
inputs = processor.apply_chat_template(
conversation, tokenize=True, return_dict=True, continue_final_message=True,
).to(model.device, model.dtype)
ids = model.generate(**inputs, max_new_tokens=256, num_beams=4)
print(processor.decode(ids[:, inputs["input_ids"].shape[1]:], return_format="transcription_only")[0])
The assistant turn is prefilled with the language tag: the processor's own language= argument checks the base
model's 30 names and rejects "Haitian Creole", the language this model adds. Requires transformers>=5.13.
Numbers come out as digits: "1500 goud", "17 desanm 2024", "25%", "32°", "5 km", "3:30", "3456-7890", "90.5". Text is lowercase without punctuation otherwise. A number below 10 on its own stays a word ("twa moun") unless money, a unit, a month or a time follows it ("5 goud", "3 me", "8è").
GGUF for llama.cpp: jsbeaudry/Oswald-ASR-0.6-V2-GGUF.
max_new_tokens by
the clip's length (about 6 tokens a second is generous for Kreyòl speech) and drop a phrase repeated more than
three times in a row.model.safetensors 1.6 GBadapter/checkpoint-4400/optimizer.pt 373 MBadapter/checkpoint-5600/optimizer.pt 373 MBadapter/adapter_model.safetensors 186 MBadapter/checkpoint-4400/adapter_model.safetensors 186 MBadapter/checkpoint-5600/adapter_model.safetensors 186 MBadapter/checkpoint-4400/rng_state.pth 14.6 kBadapter/checkpoint-5600/rng_state.pth 14.6 kBadapter/checkpoint-4400/training_args.bin 5.2 kBadapter/checkpoint-5600/training_args.bin 5.2 kBadapter/training_args.bin 5.2 kBadapter/checkpoint-4400/scheduler.pt 1.5 kBadapter/checkpoint-5600/scheduler.pt 1.5 kBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo jsbeaudry/Oswald-ASR-0.6-V2 ./model-folderQwen/Qwen3-ASR-0.6B-hf