MOSS-Transcribe-Diarize-0.9B-ONNX-INT8-ENC
ONNX export of MOSS-Transcribe-Diarize 0.9B with an INT8 audio encoder, intended for CPU inference.
Changes from the base model
Derived from soniqo/MOSS-Transcribe-Diarize-0.9B-ONNX-FP16, which is itself
an ONNX export of OpenMOSS-Team/MOSS-Transcribe-Diarize. Two modifications:
audio_encoder.onnx— weights converted FP16 to FP32, thenMatMulweights dynamically quantized to per-channel INT8.decoder.onnx— weights converted FP16 to FP32, not quantized. Stored with external data indecoder.onnx.data. Both graphs must share one numeric type, so the decoder is FP32 rather than FP16.
No weights were retrained, and no architecture, tokenizer, or configuration was altered.
Measurements
Against the FP16 bundle on the ONNX Runtime CPU provider, 24 cores, 12 intra-op threads:
| FP16 | this bundle | |
|---|---|---|
| peak resident memory | 5274 MB | 3251 MB |
| 20 s decode | 4140 ms | 3118 ms |
Accuracy measured on FLEURS test, 50 clips each of ru_ru, en_us,
it_it, cmn_hans_cn — WER for the first three, CER for Chinese. Of 200
clips, 186 produced identical transcripts, 9 improved and 5 regressed
(two-sided sign test p = 0.42).
| language | FP16 | this bundle |
|---|---|---|
| en | 5.76% | 5.48% |
| it | 5.80% | 4.82% |
| ru | 9.37% | 8.80% |
| zh (CER) | 9.99% | 9.99% |
Segment boundaries were compared separately over the same 199 clips. 127 produced byte-identical output including timestamps and activity markers, 186 produced the same number of turns, and where turns matched the boundary shift had a median of 0.00 s and a 90th percentile of 0.04 s. No turn changed whether it clears a two-second minimum, which is the threshold a downstream speaker-identity stage would care about. One clip assigned a different paragraph-local activity label.
FLEURS is single-speaker read speech, so none of this says whether speaker attribution changes on multi-speaker audio. That needs a diarization corpus and a DER comparison.
Files
audio_encoder.onnx, decoder.onnx, decoder.onnx.data, and the config,
processor_config, preprocessor_config and vocab JSON files, matching the
layout of the base repository.