This repository hosts a reproducible, hash-verified ONNX export of NVIDIA's nvidia/nemotron-3.5-asr-streaming-0.6b (checkpoint revision ea30d66debe3740a08b573244286791d423d6b3e), built and maintained by the Voice Pipes project (r2, 2026-09-23).
Nemotron 3.5 ASR Streaming 0.6B is a cache-aware streaming FastConformer-RNNT speech recognition model covering 40+ languages/locales with per-session language conditioning and automatic language detection.
Compared to the upstream PyTorch/NeMo checkpoint and multi-chunk community ONNX mirrors, this release is tailored for production voice pipelines:
lookahead = 3, frame_ms = 80.0), which sits in the sweet spot of the latency-vs-accuracy curve and keeps the total download to ~1.3 GB.encoder_320ms_first_fp16.onnx and encoder_320ms_fp16.onnx have their weights cast to FP16 using the >2 GiB-safe converter (quantize.py), cutting encoder weight storage in half (~2.4 GB → ~1.2 GB) while keeping all graph inputs and outputs in FP32.decoder.onnx and joiner.onnx remain in FP32 for numerical stability and exact token-for-token parity with the reference implementation.codavidgarcia/nemotron-3.5-asr-streaming-onnx (fe19096c852a957af1362c8facb2957b8f606c57) via scripts/nemotron35/make_export.py. Every release is gated by ONNX-vs-PyTorch parity checks and Rust token-identity tests in voice-pipes-stt-nemotron35, and every file's SHA-256 is recorded in SHA256SUMS.Note on external encoder weights:
encoder_320ms_first_fp16.onnx.dataandencoder_320ms_fp16.onnx.dataare byte-identical (1,228,512,000bytes, SHA-256159887b78f1e36fd59f5cf17be810fd61e98a96b61673b6de0dbe2b8dfb47a31). Because ONNX Runtime resolves each graph's external weights by filename, both filenames are provided in this repository forhuggingface_hubcompatibility. When using Voice Pipes'download_nemotron35.py, the 1.2 GB weights file is downloaded once and hard-linked to the second name.
| File | Size | Precision | Role |
|---|---|---|---|
encoder_320ms_first_fp16.onnx |
668 KB | FP16 weights / FP32 I/O | First-chunk FastConformer encoder graph (320 ms) |
encoder_320ms_first_fp16.onnx.data |
1.23 GB | FP16 | External weights for first-chunk encoder |
encoder_320ms_fp16.onnx |
611 KB | FP16 weights / FP32 I/O | Steady-state FastConformer encoder graph (320 ms) |
encoder_320ms_fp16.onnx.data |
1.23 GB | FP16 | External weights for steady-state encoder (byte-identical to above) |
decoder.onnx |
61.4 MB | FP32 | 2-layer LSTM RNNT prediction network (hidden_size = 640) |
joiner.onnx |
33.6 MB | FP32 | RNNT joint network (vocab_size = 13088, blank_token_id = 13087) |
nemotron_onnx_config.json |
8.3 KB | — | Model metadata, prompt_dictionary, mel feature config, and cache_shapes |
tokens.txt |
145 KB | — | SentencePiece vocabulary (13,088 entries) |
SHA256SUMS |
1 KB | — | SHA-256 checksums for all artifacts |
LICENSE-OpenMDW-1.1 |
2.6 KB | — | NVIDIA OpenMDW License Agreement v1.1 |
NOTICE |
1.0 KB | — | Provenance, attribution, and third-party notices |
Voice Pipes supports this model natively via the voice-pipes-stt-nemotron35 plugin crate (provider = "nemotron-3.5").
Download via the setup wizard (auto-downloads when selecting Nemotron 3.5) or fetch manually with SHA-256 verification:
python voice-pipes/scripts/download_nemotron35.py
Configure in pipeline.toml:
[stt]
provider = "nemotron-3.5"
model_dir = "models/nemotron-3.5"
[stt.options]
language = "en-US" # BCP-47 locale from prompt_dictionary, or "auto" (default)
# pool_size = 2 # Loaded ONNX session sets (1 per concurrent stream, ~1.3 GB each)
# threads = 4 # Intra-op threads per session (0 = ORT default)
For GPU inference, build with stt-nemotron35-cuda and set VP_ORT_EP=cuda.
huggingface_hub)hf download myned-ai/nemotron-3.5-asr-streaming-0.6b-onnx --local-dir ./nemotron-3.5-onnx
cd nemotron-3.5-onnx
sha256sum -c SHA256SUMS
Or in Python:
from huggingface_hub import snapshot_download
model_dir = snapshot_download("myned-ai/nemotron-3.5-asr-streaming-0.6b-onnx")
The model expects 16 kHz, 16-bit mono PCM audio converted to 128-bin Slaney log-mel spectrograms (n_fft=512, hop_length=160, win_length=400, preemphasis=0.97, fmin=0.0, fmax=8000.0, log_zero_guard=5.960464477539063e-08).
Each 320 ms chunk feeds the encoder (encoder_320ms_first_fp16.onnx for chunk 0, encoder_320ms_fp16.onnx for subsequent chunks) along with prompt_ids (from prompt_dictionary in nemotron_onnx_config.json), cache_mask, and the 75 recurrent cache tensors (k_cache_0..23, v_cache_0..23, conv2d_cache_0..2, conv1d_cache_0..23), followed by greedy RNNT decoding with decoder.onnx and joiner.onnx:
import json
from pathlib import Path
import numpy as np
import onnxruntime as ort
model_dir = Path("./nemotron-3.5-onnx")
cfg = json.loads((model_dir / "nemotron_onnx_config.json").read_text(encoding="utf-8"))
providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
enc_first = ort.InferenceSession(str(model_dir / "encoder_320ms_first_fp16.onnx"), providers=providers)
enc_step = ort.InferenceSession(str(model_dir / "encoder_320ms_fp16.onnx"), providers=providers)
decoder = ort.InferenceSession(str(model_dir / "decoder.onnx"), providers=providers)
joiner = ort.InferenceSession(str(model_dir / "joiner.onnx"), providers=providers)
# Select language prompt ID (e.g., "en-US" -> 0, "hr-HR" -> 29, "auto" -> 101)
prompt_id = np.array([cfg["prompt_dictionary"]["en-US"]], dtype=np.int64)
# Initialize encoder caches from cfg["cache_shapes"]
caches = {
name: np.zeros(shape, dtype=np.float32)
for name, shape in cfg["cache_shapes"].items()
}
# Initialize RNNT decoder state: [num_decoder_layers, batch=1, decoder_hidden_size]
h = np.zeros((cfg["num_decoder_layers"], 1, cfg["decoder_hidden_size"]), dtype=np.float32)
c = np.zeros((cfg["num_decoder_layers"], 1, cfg["decoder_hidden_size"]), dtype=np.float32)
last_token = np.array([[cfg["blank_token_id"]]], dtype=np.int64)
For a complete end-to-end streaming driver (including mel feature extraction, chunk framing, and RNNT decoding), see:
plugins/stt/nemotron35/src/scripts/nemotron35/vendor/export_onnx.pynemotron_onnx_config.json maps BCP-47 locales (and short aliases) to prompt_ids. Specifying an explicit locale is recommended for short utterances, as "auto" (101) can misclassify brief single-word turns.
auto (101, default_prompt_id)en-US (0), en-GB (1), es-ES (2), es-US (3), zh-CN (4), zh-TW (5), hi-IN (6), ar-AR (7), fr-FR (8), fr-CA (100), de-DE (9), ja-JP (10), ru-RU (11), pt-BR (12), pt-PT (13), ko-KR (14), it-IT (15), nl-NL (16), pl-PL (17), tr-TR (18), uk-UA (19), ro-RO (20), el-GR (21), cs-CZ (22), hu-HU (23), sv-SE (24), da-DK (25), fi-FI (26), no-NO (27), nb-NO (103), nn-NO (104), sk-SK (28), hr-HR (29), bg-BG (30), lt-LT (31), th-TH (32), vi-VN (33), id-ID (34), ms-MY (35), bn-IN (36), ur-PK (37), fa-IR (38), ta-IN (39), te-IN (40), mr-IN (41), gu-IN (42), kn-IN (43), ml-IN (44), si-LK (45), ne-NP (46), km-KH (47), sw-KE (48), am-ET (49), ha-NG (50), zu-ZA (51), yo-NG (52), ig-NG (53), af-ZA (54), rw-RW (55), so-SO (56), ny-MW (57), ln-CD (58), or-KE (59), et-EE (60), lv-LV (61), sl-SI (62), he-IL (64), ku-TR (65), az-AZ (66), ka-GE (67), hy-AM (68), uz-UZ (69), tg-TJ (70), ky-KG (71), qu-PE (80), ay-BO (81), gn-PY (82), nah-MX (83), mi-NZ (96), haw-US (97), sm-WS (98), to-TO (99), mt-MT (102).CUDAExecutionProvider.LICENSE-OpenMDW-1.1, https://openmdw.ai/license/1-1/).
Attribution: "NVIDIA Nemotron 3.5 ASR (nvidia/nemotron-3.5-asr-streaming-0.6b), NVIDIA Corporation".
This export is not affiliated with, sponsored by, or endorsed by NVIDIA.codavidgarcia/nemotron-3.5-asr-streaming-onnx at commit fe19096c852a957af1362c8facb2957b8f606c57.decoder.onnx 61 MBjoiner.onnx 34 MBencoder_320ms_first_fp16.onnx 668.4 kBencoder_320ms_fp16.onnx 610.6 kBnvidia/nemotron-3.5-asr-streaming-0.6b