Thai speech recognition for car voice commands. Say "เปิดแอร์ยี่สิบสี่องศา" or "ปิดกระจกฝั่งคนขับ" and it writes down what you said, offline, on a phone or car CPU. It's the speech front end of Lyra Car Assistant, which turns the text into a car action.
Under the hood: yfyeung/icefall-asr-gigaspeech2-th-zipformer-2024-06-20, a Thai Zipformer transducer trained with icefall on GigaSpeech2, fine-tuned here on car commands and everyday Thai. 4-bit encoder. 91 MB in all. sherpa-onnx loads it without conversion.
| file | size | |
|---|---|---|
encoder.int4.onnx + encoder.int4.onnx.data |
6.5 + 81.7 MB | Zipformer encoder; MatMuls as 4-bit MatMulNBits (blocks of 128, int8 compute) |
decoder.int8.onnx + .data |
1.3 MB | stateless decoder, int8 |
joiner.int8.onnx + .data |
1.0 MB | joiner, int8 |
tokens.txt, bpe.model |
2,000 BPE pieces |
Every weight sits in a .data file beside its graph. ONNX Runtime maps those from disk rather than copying
them, so once it has decoded a clip the model holds about 170 MB of private memory.
import numpy as np, sherpa_onnx, soundfile as sf
rec = sherpa_onnx.OfflineRecognizer.from_transducer(
encoder="encoder.int4.onnx", decoder="decoder.int8.onnx", joiner="joiner.int8.onnx",
tokens="tokens.txt", num_threads=2, decoding_method="modified_beam_search")
audio, sr = sf.read("command.wav", dtype="float32") # 16 kHz mono
lead = np.random.default_rng(0).standard_normal(sr // 2).astype(np.float32) * 1e-3
stream = rec.create_stream()
stream.accept_waveform(sr, np.concatenate([lead, audio]))
rec.decode_stream(stream)
text = stream.result.text.replace("\u0e4d\u0e32", "\u0e33") # ํา -> ำ
Three details matter:
The text comes out as Thai without spaces, and numbers always as Thai words: "ขอแอร์ยี่สิบสี่องศา", never "24". If your app wants digits, convert them yourself, and watch for words that merely contain a number word, like สามารถ or ห้าม.
Tested with sherpa-onnx 2.0.1 (Python) on macOS. sherpa-onnx also has Android, iOS and WebAssembly builds; the 4-bit encoder needs one whose ONNX Runtime knows MatMulNBits (1.17 or newer).
Character error rate, spaces ignored, numbers compared as Thai words. Lower is better.
| Gowajee test | FLEURS test (first 300) | |
|---|---|---|
| starting model, its published int8 files | 9.2% | 6.1% |
| this model | 6.3% | 5.7% |
Gowajee test: 2,717 spoken commands and requests from the corpus's 25 dev and test speakers, none of whom are in the training set. FLEURS is read, news-style Thai. It's in the table to catch the fine-tune making everyday Thai worse, which it didn't. Both were scored with the three steps above.
Three epochs over 30.1 hours of audio (23,392 clips), from the published checkpoint, with icefall's
zipformer/finetune.py. Base learning rate 0.002, 600 s of audio per batch, fp16, one RTX 4090. The training
itself took about eight minutes. The exported weights average epochs 2 and 3.
Six epochs were tried too: better on synthetic voices, worse on real ones.
| source | clips | |
|---|---|---|
| VachanaTTS, Lyra's commands | 5,286 | 1,762 Thai command sentences in its four voices, 3 renditions each |
| VachanaTTS, commands Lyra will get | 2,019 | windows, sunroof, A/C, defrost, screen, ambient light, apps, vehicle status |
| VoxCPM2 designed voices | 2,913 | 47 voices made up from text descriptions (no real person's voice is cloned) |
| joined clips | 3,000 | 2-3 short clips with pauses, so a pause isn't taken for the end |
| Gowajee train speakers | 7,998 | real people giving commands, half with car noise added |
| FLEURS Thai train | 2,176 | read Thai |
Clean TTS sounds nothing like a car, so the synthetic clips got roughed up: speed changes, cabin reverb, a cheap-mic filter, and road, A/C fan, hiss, babble or music noise at 3-25 dB SNR. Labels write numbers as Thai words and English names in Thai letters (CarPlay → คาร์เพลย์). No recordings of private individuals went in.
Apache-2.0, like the model it starts from.
e33c75c.
This model changes its weights by fine-tuning and its encoder by 4-bit quantization.@inproceedings{gigaspeech2,
title={GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinement},
author={Yifan Yang and Zheshu Song and Jianheng Zhuo and Mingyu Cui and Jinpeng Li and Bo Yang and Yexing Du and Ziyang Ma and Xunying Liu and Ziyuan Wang and Ke Li and Shuai Fan and Kai Yu and Wei-Qiang Zhang and Guoguo Chen and Xie Chen},
booktitle={Proc. ACL},
year={2025},
address={Vienna},
}
@article{fleurs2022arxiv,
title = {FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech},
author = {Conneau, Alexis and Ma, Min and Khanuja, Simran and Zhang, Yu and Axelrod, Vera and Dalmia, Siddharth and Riesa, Jason and Rivera, Clara and Bapna, Ankur},
journal={arXiv preprint arXiv:2205.12446},
year = {2022},
}
@techreport{gowajee,
title = {{Gowajee Corpus}},
author = {Ekapol Chuangsuwanich and Atiwong Suchato and Korrawe Karunratanakul and Burin Naowarat and Chompakorn CChaichot and Penpicha Sangsa-nga and Thunyathon Anutarases and Nitchakran Chaipojjana and Yuatyong Chaichana},
year = {2020},
institution = {Chulalongkorn University, Faculty of Engineering, Computer Engineering Department},
note = {Version 0.9.3},
url = {https://github.com/ekapolc/gowajee_corpus}
}
encoder.int4.onnx 7 MBdecoder.int8.onnx 4.4 kBjoiner.int8.onnx 1.5 kByfyeung/icefall-asr-gigaspeech2-th-zipformer-2024-06-20