zipformer2-66m-streaming-en · Qualcomm NPU (SM8750)
Streaming English speech recognition on the Snapdragon 8 Elite (SM8750) NPU (Hexagon v79): icefall's English streaming Zipformer2 (66M) compiled to QNN context binaries for sherpa-onnx.
Original model by Zengwei Yao and the k2-fsa team, trained with icefall (Apache-2.0) on LibriSpeech (CC-BY-4.0). sherpa-onnx publishes its ONNX export as Apache-2.0 at csukuangfj/sherpa-onnx-streaming-zipformer-en-2023-06-26. All credit for the model goes to its authors. This repo only changes the format so it runs on the NPU.
The model writes uppercase words without punctuation, as LibriSpeech's transcripts are.
Files
| File | Size | sha256 |
|---|---|---|
encoder.bin |
154,628,096 | 3eb6f7cf877ec4aa3051df7221be472064a59986823a517bd979106d31470528 |
decoder.bin |
1,163,264 | bf5a6a8d7f129cd56da9c1ccaca5941650b5e115afbcfa90024e7f9b65819591 |
joiner.bin |
581,632 | bdf0d6511d2539cf80517c398256caab3b86eec0fa873569baa4ff92921b4541 |
tokens.txt |
5,048 | 49e3c2646595fd907228b3c6787069658f67b17377c60aeb8619c4551b2316fb |
tokens.txt is the original repo's data/lang_bpe_500/tokens.txt, unchanged (identical to sherpa-onnx's).
How it was made
- Source: the full-precision checkpoint
exp/pretrained.ptfromZengwei/icefall-asr-librispeech-streaming-zipformer-2023-05-17at commit37cb5606(sha256e44bb7c8…85342), not an int8 ONNX file. - ONNX export: icefall's
zipformer/export-onnx-streaming.py(icefall commit9c96676), with fixed shapes and no int8 quantization:--use-int32-inputs 1 --dynamic-batch 0 --enable-int8-quantization 0 --causal 1 --chunk-size 64 --left-context-frames 256and the model's sizes (--num-encoder-layers 2,2,3,4,3,2 --feedforward-dim 512,768,1024,1536,1024,768 --encoder-dim 192,256,384,512,384,256 --encoder-unmasked-dim 192,192,256,256,256,192). Chunk 64 is 1.28 s of audio. - QNN compile: Qualcomm AI Runtime (QAIRT) 2.40.0.251030, 16-bit activations and 32-bit bias
(
qnn-onnx-converter --act_bitwidth 16 --bias_bitwidth 32), thenqnn-context-binary-generatorfor SM8750 with sherpa-onnx'sscripts/qnn/generate_config.py. Graph names areencoder,decoderandjoiner.
Use
The context binaries are tied to the chip (SM8750) and to the QAIRT version: load them with the 2.40.0.251030 QNN
runtime libraries (libQnnHtp.so, libQnnHtpV79Skel.so, libQnnHtpV79Stub.so, libQnnSystem.so; sherpa-onnx
publishes them as qnn-libs-2.40.0.251030). Point ADSP_LIBRARY_PATH at their folder.
With sherpa-onnx's CLI on the phone (adb shell):
export ADSP_LIBRARY_PATH=$PWD
./sherpa-onnx --provider=qnn --tokens=tokens.txt \
--transducer.qnn-backend-lib=libQnnHtp.so --transducer.qnn-system-lib=libQnnSystem.so \
--transducer.qnn-context-binary=encoder.bin,decoder.bin,joiner.bin \
test.wav
Input: 16 kHz mono audio, 80-dim fbank features, as the original model.
Results
- ONNX export: decoded on a PC with the exported graphs, it gives the same text as sherpa-onnx's int8 model on its LibriSpeech test clips.
- The same pipeline built the Hebrew model's NPU build from this model's architecture; on a Galaxy S25 that one decoded 30 s of audio in 2.2 s for one stream.
License and credit
Apache-2.0, as the original's ONNX release. If you use these files, credit the original: "icefall streaming Zipformer by Zengwei Yao, k2-fsa (Apache-2.0), trained on LibriSpeech (CC-BY-4.0)". Changes from the original: exported to ONNX with fixed shapes and compiled to Qualcomm QNN context binaries, 16-bit, for SM8750.
The QNN runtime libraries needed to run these files are Qualcomm's and are not included here.