VibeVoice-ASR-Streaming 7B — ONNX export for Vernacula
An ONNX export of microsoft/VibeVoice-ASR-Streaming-7B,
Microsoft's streaming speaker-attributed ASR model (Qwen2.5-7B decoder over a pair of causal
audio tokenizers), for use as an ASR backend in
Vernacula.
The model transcribes who said what as audio arrives, with no separate diarizer: it emits one text chunk per 2.93 s of audio and marks speaker turns inline.
- Conversion scripts:
scripts/vibevoice_streaming_export/ - Investigation log:
docs/dev/vibevoice_asr_streaming_investigation.md
Files
| file | what it is |
|---|---|
audio_encoder.onnx (+ .data) |
acoustic + semantic tokenizer encoders and both connectors, float16. One fixed 83,200-sample window (3.47 s) in, 26 frames out. |
decoder_gqa.onnx (+ .data) |
Qwen2.5-7B decoder, INT8 weight-only, attention as com.microsoft::GroupQueryAttention over a shared float16 KV cache. |
export-report.json |
shapes, the streaming constants, and the prompt/special token ids the runtime needs. |
tokenizer.json, config.json, preprocessor_config.json, tokenizer_config.json |
metadata carried through from the source checkpoint. |
How it differs from the source checkpoint
- Deterministic audio encoding. Upstream samples the acoustic latents with Gaussian noise; this export uses the latent mean, which is bit-repeatable and sits inside the seed-to-seed variation of the original.
- INT8 decoder weights. Measured indistinguishable from float16 on transcript accuracy, while halving the weights.
- Shared, pre-allocated KV cache. VRAM is flat in recording length rather than growing, and throughput does not decay over a long file.
Requirements and limits
- ONNX Runtime with the CUDA execution provider.
GroupQueryAttentionand the float16 graphs are not supported on the CPU provider here. - Peak VRAM about 15.7 GB; real-time factor about 0.142 on an RTX 3090.
- The KV cache ceiling in
export-report.jsonbounds recording length and is set to the checkpoint's trained context (131,072 positions, roughly 2 hours of audio). The runtime refuses a longer recording rather than failing partway. Attention cost grows with the filled cache, so decoding slows as a long recording proceeds; VRAM does not. - Speaker attribution at this size is reliable — it separated two speakers correctly across a 30-minute file.
Licence
MIT, matching the source checkpoint.