Haas-ja-v1.0-onnx-quantized
A quantized, CPU-friendly ONNX conversion of third-intelligence/Haas-ja-v1.0, a Japanese speech recognition model. It runs on unmodified sherpa-onnx, with no PyTorch, NeMo, Triton or GPU.
This is an unofficial community conversion, not made or endorsed by Third Intelligence. The original model officially supports only Linux with an NVIDIA GPU. For the model itself, its training data and its published benchmark, see the original model card.
An unquantized version is available as Haas-ja-v1.0-onnx. It is 5 times larger and needs about 3 GB of memory, with similar accuracy.
Files
| File | Size | Contents |
|---|---|---|
encoder.quantized.onnx + encoder.data |
432 MB | FastConformer encoder, quantized (see below). Keep both files in one folder. |
decoder.onnx |
15 MB | Kimi Delta Attention prediction network, float32 |
joiner.onnx |
12 MB | Joint network, float32 |
tokens.txt |
3,072 tokens plus <blk> |
Usage
Tested with sherpa-onnx 1.13.6 on an Apple M1 Pro.
import sherpa_onnx
import soundfile as sf
recognizer = sherpa_onnx.OfflineRecognizer.from_transducer(
encoder="encoder.quantized.onnx",
decoder="decoder.onnx",
joiner="joiner.onnx",
tokens="tokens.txt",
num_threads=4,
model_type="nemo_transducer",
feature_dim=128,
)
samples, sample_rate = sf.read("example.wav", dtype="float32") # mono
stream = recognizer.create_stream()
stream.accept_waveform(sample_rate, samples)
recognizer.decode_stream(stream)
print(stream.result.text)
Measurements
Apple M1 Pro, 4 threads, sherpa-onnx 1.13.6, one clip at a time.
This is not the original card's metric. CER here is plain edit distance after NFKC and removing punctuation and spaces, with no kanji/kana or numeral normalization. It cannot be compared with the 6.12% on the original card, and the original results have not been reproduced with jacer. "Original (PyTorch)" is the original weights run in float32 on CPU through a pure-PyTorch replacement for the Triton kernels, not the official GPU setup.
| Dataset | Clips | This conversion: CER | This conversion: RTF | Original (PyTorch): CER |
|---|---|---|---|---|
| FLEURS ja (test, asr-ja-bench v1.1) | 650 | 4.34% | 0.039 | 4.19% |
| ReazonSpeech test (random 2.7 h) | 1,645 | 5.54% | 0.041 | 5.57% |
| SpeechBSD ja (test, asr-ja-bench v1.1) | 2,093 | 5.96% | 0.040 | 6.06% |
The differences from the original are not statistically clear: the 95% bootstrap intervals include zero on all three datasets. Quantization is not expected to improve accuracy, so treat the small gains in the table as noise. ReazonSpeech is part of the original model's training corpus, so that row may be optimistic even though it uses the test split.
Memory (peak process RSS):
| Longest clip | Peak |
|---|---|
| 12 s | about 0.80 GB |
| 19–29 s | 0.97–1.12 GB |
Memory grows with clip length, so split long audio into segments, for example with a VAD. Only clips up to about 29 seconds have been tested.
How it was made
- Encoder: exported with NeMo's exporter from float32 weights, then pointwise convolutions were quantized to int8 (
quantize_dynamic, excluding depthwise convolutions) and MatMul weights to 4-bit (MatMulNBitsQuantizer, block size 32, symmetric, accuracy level 4). Before quantization, ONNX and PyTorch encoder outputs matched within about 2e-5. - Decoder and joint: exported from the original weights. An earlier export with explicit state matched PyTorch within 7e-6 (prediction network) and 5e-5 (joint) over a 61-step label sequence, and this package's decoder matched that export exactly over 41 steps. This package's joiner was checked only through the end-to-end results above.
- Conversion scripts are in
conversion/.
Compatibility notes
sherpa-onnx has no dedicated support for this model. It loads through the Parakeet TDT path, which relies on two details that a future sherpa-onnx version could change:
- The Kimi Delta Attention state is flattened into two
[2, B, 20480]tensors that sherpa-onnx passes around as LSTM state, so the encoder metadata sayspred_hidden=20480. The prediction network's real hidden size is 640. - sherpa-onnx detects TDT models by looking for
tdtin the metadataurl, so the URL ends in#tdt.
Limitations
- Features differ from NeMo. sherpa-onnx computes its own log-mel features, so transcripts are not always identical to the original model's, even when the error rate is similar.
- Offline only. Like the original, the model sees a whole segment at a time.
- Tested on one machine. Other CPUs, thread counts and sherpa-onnx versions have not been measured. Long audio, custom vocabulary and speaker labels have not been tested.
- The limitations in the original model card also apply.
License and attribution
Apache License 2.0, the same as the original model. See LICENSE and NOTICE.
Please credit and cite the original work by Third Intelligence. Its acknowledgements cover the datasets and projects it builds on, including NVIDIA NeMo, Parakeet-TDT, Kimi Linear and flash-linear-attention.