Nemotron 3 Diarization, ONNX int8 (for echo-minutes)
A dynamic int8 quantisation of the ONNX export of NVIDIA's Nemotron 3 Diarization (Streaming Sortformer v3). The echo-minutes app uses it as its crosstalk detector.
- Model, weights and licence: NVIDIA's. This is not an official NVIDIA release. See NVIDIA's model card for the description, training data, evaluation, and the bias, safety and privacy notes.
- ONNX export:
altunenes/parakeet-rs,nemotron-3-diarization/nemotron3_diar_v3.onnx, revision664cc742197053fa11afd12f5100749fb8000181, sha256c32d6056f1ca52bdd50a74e1e7c66d91e5147bc29f19ffa4c39cbe5452397d15. - Quantisation:
onnxruntime.quantization.quantize_dynamic(onnxruntime 1.32.0), with QInt8 weights for MatMul and Gemm. The recipe isquant.pyin the echo-minutes repository (workflow/changes/nemotron-int8/).
Files
nemotron3_diar_v3_int8.onnx- 107,437,992 bytes
- sha256
62936c5aa9d7175d707bffd75fe010bf9c79d86e9cb0abcfbca361b66b74873d
LICENSE: OpenMDW-1.1, the licence of the original model.
Check against fp32
These numbers come from one 19-minute meeting on Apple M4 Pro, with ONNX Runtime 1.31.0 on CPU:
- int8 and fp32 mark the same crosstalk.
- int8 is about 2× faster than fp32.
- int8 is 107 MB instead of 402 MB.
Licence
OpenMDW License Agreement, version 1.1, the same as the original model. A copy is in LICENSE.