Streaming English speech recognition on the Snapdragon 8 Elite (SM8750) NPU (Hexagon v79): icefall's English streaming Zipformer2 (66M) compiled to QNN context binaries for sherpa-onnx.
Original model by Zengwei Yao and the k2-fsa team, trained with icefall (Apache-2.0) on LibriSpeech (CC-BY-4.0). sherpa-onnx publishes its ONNX export as Apache-2.0 at csukuangfj/sherpa-onnx-streaming-zipformer-en-2023-06-26. All credit for the model goes to its authors. This repo only changes the format so it runs on the NPU.
The model writes uppercase words without punctuation, as LibriSpeech's transcripts are.
| File | Size | sha256 |
|---|---|---|
encoder.bin |
154,628,096 | 3eb6f7cf877ec4aa3051df7221be472064a59986823a517bd979106d31470528 |
decoder.bin |
1,163,264 | bf5a6a8d7f129cd56da9c1ccaca5941650b5e115afbcfa90024e7f9b65819591 |
joiner.bin |
581,632 | bdf0d6511d2539cf80517c398256caab3b86eec0fa873569baa4ff92921b4541 |
tokens.txt |
5,048 | 49e3c2646595fd907228b3c6787069658f67b17377c60aeb8619c4551b2316fb |
tokens.txt is the original repo's data/lang_bpe_500/tokens.txt, unchanged (identical to sherpa-onnx's).
exp/pretrained.pt from
Zengwei/icefall-asr-librispeech-streaming-zipformer-2023-05-17 at commit
37cb5606
(sha256 e44bb7c8…85342), not an int8 ONNX file.zipformer/export-onnx-streaming.py (icefall commit 9c96676), with fixed shapes and
no int8 quantization: --use-int32-inputs 1 --dynamic-batch 0 --enable-int8-quantization 0 --causal 1 --chunk-size 64 --left-context-frames 256 and the model's sizes (--num-encoder-layers 2,2,3,4,3,2 --feedforward-dim 512,768,1024,1536,1024,768 --encoder-dim 192,256,384,512,384,256 --encoder-unmasked-dim 192,192,256,256,256,192). Chunk 64 is 1.28 s of audio.qnn-onnx-converter --act_bitwidth 16 --bias_bitwidth 32), then qnn-context-binary-generator for SM8750 with
sherpa-onnx's scripts/qnn/generate_config.py. Graph names are encoder, decoder and joiner.The context binaries are tied to the chip (SM8750) and to the QAIRT version: load them with the 2.40.0.251030 QNN
runtime libraries (libQnnHtp.so, libQnnHtpV79Skel.so, libQnnHtpV79Stub.so, libQnnSystem.so; sherpa-onnx
publishes them as qnn-libs-2.40.0.251030). Point ADSP_LIBRARY_PATH at their folder.
With sherpa-onnx's CLI on the phone (adb shell):
export ADSP_LIBRARY_PATH=$PWD
./sherpa-onnx --provider=qnn --tokens=tokens.txt \
--transducer.qnn-backend-lib=libQnnHtp.so --transducer.qnn-system-lib=libQnnSystem.so \
--transducer.qnn-context-binary=encoder.bin,decoder.bin,joiner.bin \
test.wav
Input: 16 kHz mono audio, 80-dim fbank features, as the original model.
Apache-2.0, as the original's ONNX release. If you use these files, credit the original: "icefall streaming Zipformer by Zengwei Yao, k2-fsa (Apache-2.0), trained on LibriSpeech (CC-BY-4.0)". Changes from the original: exported to ONNX with fixed shapes and compiled to Qualcomm QNN context binaries, 16-bit, for SM8750.
The QNN runtime libraries needed to run these files are Qualcomm's and are not included here.
encoder.bin 155 MBdecoder.bin 1 MBjoiner.bin 581.6 kBZengwei/icefall-asr-librispeech-streaming-zipformer-2023-05-17