SenseVoice-small on LiteRT (dynamic-int8, bucketed)
FunAudioLLM/SenseVoiceSmall converted
PyTorch → LiteRT via litert-torch, quantized with ai-edge-quantizer dynamic_wi8_afp32.
Built for VoxSumDroid's CPU/XNNPACK path (LiteRT ≥ 2.1.6).
Contract
Multi-signature bucket export — signatures sv_63 / sv_125 / sv_250 / sv_500
(max LFR frames ≈ 3.8 / 7.5 / 15 / 30 s of audio). Inputs (args_0..3):
| input | shape | dtype | meaning |
|---|---|---|---|
| args_0 | [1, T_bucket, 560] | f32 | LFR(7,6)-stacked, CMVN-normalized kaldi fbank |
| args_1 | [1] | i32 | true LFR length (padding is masked out) |
| args_2 | [1] | i32 | language id: auto 0, zh 3, en 4, yue 7, ja 11, ko 12 |
| args_3 | [1] | i32 | textnorm id: withitn 14, woitn 15 |
Output: CTC logits [1, T_bucket+4, 25055] (4 prompt rows first; blank id 0;
valid rows = args_1 + 4; one row = 60 ms). tokens.txt maps ids to sentencepiece
pieces (identical to the official bpe model, verified 0/25055 mismatches);
cmvn.json holds the 560-d shift/scale from the official am.mvn.
Front end: kaldi fbank, 25/10 ms, hamming, 80 bins (20 Hz–8 kHz kaldi mel), dither 0, waveform ×32768, DC removal, preemphasis 0.97, snip_edges, natural log.
Validation (vs the PyTorch checkpoint, host CPU)
- fp32 tflite: worst element-wise max|Δ| 8.0e-5 (real audio) / 4.1e-4 (synthetic battery over all buckets, padded and full), CTC text exact on the official zh/en/yue example clips.
- this q8: logits cosine ≈ 0.9985; long-audio CER vs fp32 ground truth, compared with the sherpa-onnx int8 export on identical features:
| clip | int8 ONNX | this q8 |
|---|---|---|
| zh meeting, 3 min | 4.78% | 2.33% |
| zh healthcare, 6 min | 3.55% | 2.54% |
| AMI EN2002a, 6 min | 4.24% | 3.29% |
- ARM (RPi4, XNNPACK): decoded token ids identical to x86 on en/yue; zh differs by one known knife-edge token ("五点"↔"5点").
- int4 (
dynamic_wi4_afp32) was evaluated and REJECTED: CER ≈ 21% on all clips.
Tensor names are stripped (−5.5 MB); signature and I/O names are kept.