cabin-asr-vi-zipformer-offline-r2 (release candidate rc1)
Vietnamese offline (non-streaming) zipformer2 transducer, 68.6 M parameters, BPE-2000, for the Cabin Offline app. Same architecture and file names as the shipped sherpa-onnx-zipformer-vi-int8-2025-04-20 (base zzasdf/viet_iter3_pseudo_label). Fine-tuned from pilot1 on transcripts written by R13x (Qwen3-ASR-1.7B fine-tune, Apache-2.0) plus human read speech.
- Status: release candidate
rc1(2026-10-29): drop-in replacement for the shipped Vietnamese model, round 2 of the R13x → sherpa-vi distillation (trained 2026-10-24). The int8 package passed the host release check below. Still unmeasured: real phone captures (private, Mac only), the app's invented-card suite and on-device (ARM) RTF; those gate the switch from release candidate to shipped. - Collection: Cabin offline ASR (on-device, distilled from R13x). Repo names:
doanhlt/cabin-asr-<lang>-<arch>-<variant>-<version>. - Code:
thinkprompt/cabin-offline-asr(private GitHub repo), branchfeat/mobile-local-kit,mobile-local/distill/+mobile-local/README.md("Pilot: R13x → sherpa-vi distillation"); round-2 queue in commit1548525, this upload script in the commit that addsdistill/hub_publish.py.
Files
| path | what |
|---|---|
package/ |
the sherpa-onnx package the app loads (encoder / joiner int8, decoder fp32, tokens.txt, bpe.model) |
icefall/pretrained.pt |
icefall state dict (epoch 3, avg 2): start point for the next fine-tune (--finetune-ckpt) |
eval/ |
paired-bootstrap comparisons (vs shipped, pilot1, r2human control), round-2 gate, publish gate, dev-pick log, mix stats |
package-fp32/ |
fp32 export of the same weights (encoder / decoder / joiner fp32): reference for the int8 check, not for the phone |
eval/release_check/ |
rc1 release check: int8 vs fp32 vs shipped per utterance bootstrap, spot checks, CPU RTF (eval/r2pkg_eval.py) |
data/r2_cs_train.jsonl |
the code-switch training rows (clip id, R13x label, R11c label, agreement) |
data/yodas2_vi_heldout_*_clean.jsonl |
the two clean conversational held-out sets (refs) |
attribution/ |
YouTube video ids of all YODAS training audio (CC BY 3.0) |
Every file is listed with its sha256 in SHA256SUMS.txt (sha256sum -c SHA256SUMS.txt) and with its size in FILES.tsv.
Results (WER %, lower is better; 95 % paired bootstrap over utterances, 10,000 resamples; * = CI excludes 0)
| set (n) | shipped | pilot1 | this | this − shipped | this − pilot1 |
|---|---|---|---|---|---|
| FLEURS-100 vi (gate <= 10.22) (100) | 10.19 | 7.91 | 7.44 | -2.75 [-4.62, -1.39] * | -0.48 [-1.10, +0.07] n.s. |
| CV13-vi test200 (200) | 9.58 | 9.11 | 8.37 | -1.21 [-2.46, +0.00] n.s. | -0.74 [-1.81, +0.21] n.s. |
| YODAS2 held-out clean, ref = R13x ∩ caption (463) | 7.43 | 5.73 | 5.61 | -1.82 [-2.21, -1.46] * | -0.12 [-0.35, +0.12] n.s. |
| YODAS2 held-out clean, ref = caption (899) | 52.59 | 51.74 | 51.78 | -0.81 [-1.12, -0.51] * | +0.03 [-0.15, +0.22] n.s. |
Table above: candidate measurement of 2026-10-24 (int8 package, 4 CPU threads); the release check below repeats it at 1 / 2 / 4 threads.
Publish gate (eval/gate_publish.json): fleurs100_le_10.22 = pass, sig_better_than_shipped_on_ge_2_sets = pass, not_sig_worse_than_previous_on_any_set = pass, size_within_5pct_of_shipped = pass, rtf_within_10pct_of_shipped = pass. Package 79.3 MB vs shipped 77.4 MB (same graph; the int8 encoder is +2.5 % from re-quantisation, like pilot1); CPU RTF 4 threads 0.022 vs 0.024.
Honest reading: r2 is significantly better than the shipped model on FLEURS-100 and on both clean YODAS2 held-out sets (CV13: −1.21, CI touches 0). Against pilot1 every difference is within noise (FLEURS −0.48 and CV13 −0.74 lean better, both CIs include 0), so the round-2 script gate (eval/gate_r2.json, check "beats pilot1 somewhere") failed. It is published as the next candidate because it is never worse than pilot1, adds code-switch and human read speech, and is the start point of the next round. Held-out "clean" = YODAS2 held-out videos that are not in the R11c YODAS pool, so also out of R13x's own training data. pilot1's card used the full held-out sets (1,125 / 1,660 rows); there r2 scores 4.41 / 37.24 vs pilot1 4.59 / 37.33 (r2: eval/pick_r2.log, pilot1: its card; not bootstrapped).
Release check (rc1, 2026-10-29)
The package/ files are unchanged since the candidate upload (sha256 equal; re-exported from icefall/pretrained.pt they are
byte-identical). Built exactly like the app builds its Vietnamese recognizer: OfflineRecognizer.from_transducer, greedy search,
16 kHz, 80-dim fbank, encoder / joiner int8, decoder fp32. package-fp32/ is the fp32 export of the same weights (reference only).
| set (n) | shipped int8 | r2 fp32 | r2 int8 | int8 − fp32 | int8 − shipped |
|---|---|---|---|---|---|
| FLEURS-100 vi (gate <= 10.22) (100) | 10.19 | 7.47 | 7.98 | +0.51 [-0.10, +1.60] n.s. | -2.21 [-3.27, -1.26] * |
| CV13-vi test200 (200) | 9.58 | 8.43 | 8.37 | -0.07 [-0.32, +0.14] n.s. | -1.21 [-2.46, +0.00] n.s. |
| YODAS2 held-out clean, ref = R13x ∩ caption (463) | 7.43 | 5.66 | 5.61 | -0.04 [-0.15, +0.07] n.s. | -1.82 [-2.21, -1.46] * |
| YODAS2 held-out clean, ref = caption (899) | 52.58 | 51.77 | 51.76 | -0.01 [-0.09, +0.08] n.s. | -0.82 [-1.13, -0.52] * |
Table at 2 threads (1 thread is identical). int8 − fp32 per set is at most +0.51 WER: the +0.30 budget holds on CV13, agree-clean, caption-clean and pooled, and is exceeded on FLEURS-100 at 1–2 threads only (CI includes 0; one utterance, see "Threads"); 80–96 % of utterances get the identical text.
Threads. ONNX Runtime's x86 int8 kernels are not bit-identical across intra-op thread counts (fp32 is). At 4 threads r2 int8 scores FLEURS-100 7.44 (CV13 8.37, agree 5.61, caption 51.78); 1 thread = 2 threads exactly. The FLEURS gap at 1–2 threads is one utterance whose greedy path drops a 10-word stretch (the shipped model drops it at every thread count). Pooled over FLEURS + CV13 + agree-clean, int8 − fp32 = +0.05 [-0.10, +0.26] n.s. at 2 threads and -0.04 [-0.13, +0.04] n.s. at 4. Probes, not shipped: QUInt8 weights are thread-stable (FLEURS 7.47) but caption-clean +0.13 [+0.04, +0.22] * vs fp32; per-channel QInt8 moves the flip to 4 threads. Phones run ARM int8 kernels: compare device output against these hyps.
Spot checks (numbers, English loanwords / foreign names = tokens that are not a well-formed Vietnamese syllable, short clips):
| subset (pooled FLEURS-100 + CV13 + agree-clean) | n | shipped | r2 int8 | int8 − shipped | empty output ship / r2 |
|---|---|---|---|---|---|
| ref_has_digits | 13 | 25.42 | 21.10 | -4.32 [-8.29, -1.03] * | 0 / 0 |
| ref_has_foreign_word | 61 | 17.36 | 13.88 | -3.48 [-5.24, -1.85] * | 0 / 0 |
| short_le_2s | 8 | 9.68 | 9.68 | +0.00 [+0.00, +0.00] n.s. | 0 / 0 |
| short_le_3_words | 13 | 20.69 | 10.34 | -10.34 [-29.63, +7.14] n.s. | 0 / 0 |
Output format is the shipped model's: same tokens.txt / bpe.model (byte-identical), upper case, no punctuation, numbers
as words (all upper: True, outputs with digits 0, with punctuation 0, of 1662).
CPU real-time factor (decode wall / audio, interleaved, median of 3 rounds, 120 clips / 1021.0 s, AMD EPYC 7313 16-Core Processor; x86, not a phone; ARM numbers still to be measured on Android):
| threads | shipped int8 | r2 int8 | r2 fp32 | median latency, clips ≤ 2 s, ms (ship / r2) |
|---|---|---|---|---|
| 1 | 0.0234 | 0.0234 | 0.0317 | 42 / 42 |
| 4 | 0.0176 | 0.0183 | 0.0215 | 36 / 36 |
Size: package/ 79.3 MB vs shipped 77.4 MB (+2.5 %); fp32 272.7 MB.
Full report: eval/release_check/REPORT.md. Pin downloads to the rc1 tag (revision="rc1").
Training data, licences and attribution
Total 96.01 h, 51 % human-labelled. Started from pilot1, so pilot1's data is included.
| part | hours | labels | licence |
|---|---|---|---|
YODAS2 vi000 clips (pilot1 rows, espnet/yodas2) |
27.646 | R13x, kept where it agrees with the YouTube caption | CC BY 3.0 |
YODAS (v1) real code-switch clips via the R11c mix (espnet/yodas), data/r2_cs_train.jsonl |
19.019 (12,272 clips) | R13x relabel, kept where it agrees with the R11c label (interior WER ≤ 0.3) | CC BY 3.0 |
| FOSD (FPT Open Speech Dataset) + Common Voice 13 vi, via R11c (noise-augmented SNR 5–20 dB with MUSAN noise + ESC-10) | 35.689 | human | CC BY 4.0 (FOSD), CC0 (CV13); noise CC BY 4.0 / CC BY |
FLEURS vi train ×3 replay (google/fleurs) |
13.657 | human | CC BY 4.0 |
Attribution:
- YODAS / YODAS2 (Li et al., ASRU 2023; espnet): CC BY 3.0. The YouTube video ids of every training clip are in
attribution/yodas2_video_ids.txt(pilot1 rows) andattribution/yodas_r11c_video_ids.txt(code-switch rows). - FPT Open Speech Dataset (FOSD), © FPT Corporation, CC BY 4.0 (Mendeley Data).
- Mozilla Common Voice 13.0 vi: CC0. CV13 rows whose sentence is in our CV13 test200 eval set were dropped.
- FLEURS (Conneau et al., 2022, Google): CC BY 4.0.
- MUSAN noise (Snyder et al., 2015): CC BY 4.0; ESC-10 (Piczak, 2015): CC BY. Used only as additive noise.
Licence: Apache-2.0. Base zzasdf/viet_iter3_pseudo_label (the shipped app model) is Apache-2.0; the teacher R13x
(R13-1.7B RC1, a Qwen3-ASR-1.7B fine-tune) is Apache-2.0. No Gemini, Soniox or OpenAI output is used anywhere (labels,
filters or text), no user capture, no eval set: FLEURS test/dev, CV13 test200 and the YODAS2 dev / held-out videos are
excluded from training (video-disjoint splits). Not measured yet on real phone captures.
Recipe
- icefall zipformer
finetune.py --do-finetune 1from pilot1'spretrained.pt; LR 0.001 (Eden lr_epochs 100, lr_batches 100k), 4 epochs, max-duration 200, fp16, one A100 shared with other jobs (10 GB hard cap), one seed. - Picked on dev only (FLEURS-vi dev + YODAS2-vi dev, mean WER) among (epoch, avg) 4:2 4:3 3:2 2:1: epoch 3, avg 2.
- Control arm r2human (same, without the code-switch rows): r2 beats it on FLEURS-100 (−0.68 [−1.24, −0.22]), the other
three sets are n.s. (
eval/compare_*r2human*). - Export: icefall
export-onnx.py, encoder / joiner int8, decoder fp32, non-causal;icefall/pretrained.ptexported from the same averaged model gives byte-identical ONNX files.
How to use (sherpa-onnx, Python)
# pip install sherpa-onnx soundfile huggingface_hub
import sherpa_onnx, soundfile as sf
from huggingface_hub import snapshot_download
d = snapshot_download("doanhlt/cabin-asr-vi-zipformer-offline-r2", allow_patterns=["package/*"], revision="rc1") + "/package"
rec = sherpa_onnx.OfflineRecognizer.from_transducer(
encoder=f"{d}/encoder-epoch-12-avg-8.int8.onnx", decoder=f"{d}/decoder-epoch-12-avg-8.onnx",
joiner=f"{d}/joiner-epoch-12-avg-8.int8.onnx", tokens=f"{d}/tokens.txt",
num_threads=4, decoding_method="greedy_search") # greedy = what the app runs
audio, sr = sf.read("clip.wav", dtype="float32") # mono; sherpa-onnx resamples to 16 kHz
s = rec.create_stream(); s.accept_waveform(sr, audio); rec.decode_stream(s)
print(s.result.text)
The file names are those of the shipped sherpa-onnx-zipformer-vi-int8-2025-04-20, so the app swaps the model by
directory only.