Duyu-small
A Turkish + English speech recognizer for CPUs and edge devices: 19.3 M active parameters, shipped as ONNX graphs at int8 (56 MB) and 4-bit (34 MB) with a small torch-free engine (numpy + onnxruntime + sentencepiece). It writes whatever is spoken, Turkish, English or both in one sentence, cased and punctuated.
Duyu-small is built for on-device use: live, short, two-language speech, where the model runs next to the microphone. It is not meant for long-form transcription.
Architecture
| part | what | stored params |
|---|---|---|
| encoder | 9 Conformer blocks (d 256, 4 heads, conv k15) applied in 3 shared refinement passes with learned per-pass gates, 25 Hz output | 15.5 M |
| sparse acoustic memory | product-key memory after block 4: 250k × 120 values, 4 heads × 16 reads per frame per pass | 30.0 M (23k read per frame) |
| CTC head | 2048 SentencePiece pieces (TR + EN), InterCTC + self-conditioning at block 4 of every pass | 0.5 M |
| decoder | 3-layer Transformer (d 256, QK-norm), joint CTC/attention decoding | 3.8 M |
Active per frame: 19.3 M. Stored: 49.3 M. Compute-equivalent of a 46.6 M single-pass model.
Accuracy
Word error rate (wer_trnum: case-, punctuation- and number-format-blind),
PyTorch reference decode (attention beam 5 + CTC rescoring).
| test set | Duyu-small (49.3 M) | whisper-base (74 M) | mihuai/turkish-stt (66 M) | balbal (45.4 M) | Cactus Whistle |
|---|---|---|---|---|---|
| voicedata-TR (conversation, 235 chunks) | 12.45 | 29.51 | 19.89 | — | — |
| FLEURS tr test (743) | 7.87 | 24.19 | 9.62 | 7.95 | — |
sierra-tr (call-centre turns, 846, wer_sierra) |
16.37 | 39.08 | 24.95 | 17.61 | — |
| FLEURS en_us test (500) | 22.88 | 10.81 | Turkish only | Turkish only | 14.37 |
whisper-base and Cactus Whistle decoded with the language forced to English on
FLEURS-en and to Turkish on the Turkish sets; Duyu-small with <en> on
FLEURS-en and <tr> elsewhere.
Quantization check on 62 held-out conversational dev clips (13.7 min, the engine on a laptop CPU with 4 threads, benchmark numerics):
| engine | WER | real-time factor |
|---|---|---|
| PyTorch reference (GPU, bf16) | 20.14 | — |
| fp32, attention beam 5 | 19.56 | 0.11 |
| int8, attention beam 5 | 19.56 | 0.07 |
| 4-bit, attention beam 5 | 19.67 | 0.13 |
| int8, CTC greedy only | 22.19 | 0.016 |
The engine's beam search is a port of the reference PyTorch decoder and is verified identical to it on 20/20 dev clips at matching numerics.
Files
| path | size | purpose |
|---|---|---|
int8/, q4/, fp32/ |
56 / 34 / 201 MB per set | one complete set of the four graphs per precision; DuyuSmall(quant=...) picks the folder |
*/encoder_pre.onnx |
1.3 MB | fbank → subsampler + style vector |
*/encoder_pass.onnx |
181 / 47.6 / 26.9 MB | one refinement pass over the shared blocks, incl. the memory (fp32 / uint8 / packed 4-bit); run 3× |
*/encoder_post.onnx |
2.1 / 0.5 / 0.3 MB | CTC head |
*/decoder.onnx |
17.4 / 6.4 / 5.0 MB | attention decoder (no KV cache) |
passes.npz |
— | pass embeddings + gates |
fbank.npz, spm.model, engine.json, config.json |
— | feature tables, tokenizer, special ids, model config |
duyu_small/ |
— | the engine (Python package) |
int8 = onnxruntime dynamic quantization of every MatMul + uint8 memory table with fp16 per-row scales. q4 = 4-bit block-wise (32) MatMul weights + the memory table packed two values per byte, dequantized after the gather with plain ONNX ops (Mod / Div / Reshape), so it runs on stock onnxruntime.
Use
pip install -e . # or: pip install numpy onnxruntime sentencepiece
duyu-small recording.m4a # int8, attention beam 5, writes whatever is spoken
duyu-small a.wav --quant q4 --lang tr # 4-bit, force Turkish
duyu-small a.wav --mode ctc # CTC greedy only (partials / fastest)
from duyu_small import DuyuSmall
m = DuyuSmall() # int8 by default; quant="q4" | "fp32"
text = m.transcribe(wav16k) # float32 numpy, any length (chunked above 30 s)
text = m.transcribe(wav16k, lang="en", beam=1)
DuyuSmall() finds the model files next to the package (this repository's
layout); from elsewhere pass the directory, DuyuSmall("/path/to/Duyu-small"),
or set DUYU_SMALL_DIR.
Audio in: 16 kHz mono float32 (duyu_small.audio.load_audio reads anything
ffmpeg reads). Languages: tr, en, mix (default), auto.
Limits
- Full-context encoder: utterances are decoded whole (chunked at 25 s on long input); no streaming partials beyond the CTC head yet.
- The decoder re-runs the prefix at every step (no KV cache): fine for conversation turns, not tuned for long dictation.
- English is "understands English in a Turkish conversation" (FLEURS-en 22.9), not Whistle-class English.
- Read speech with rare proper nouns is where it is weakest.
License
Apache License 2.0: model weights, ONNX graphs and the engine.
Credits
The encoder design (a Conformer applied in 3 shared refinement passes with a sparse acoustic memory of 250k × 120 values, read by 4 heads × 16 records per pass) is taken from balbal, described by its author at erayy.com/balbal-blog. Duyu-small adds the attention decoder and the bilingual Turkish + English vocabulary on top.