Nemotron-3.5-ASR-Streaming-0.6B GGUF

GGUF conversion of nvidia/nemotron-3.5-asr-streaming-0.6b for use with CrispASR.

Model details

Architecture: Cache-Aware Streaming FastConformer encoder (24 layers, d=1024, 8 heads) + RNN-T decoder (2-layer LSTM, hidden=640) + joint network (640 → 13088 vocab).

Languages: 39 languages, selected via prompt_kernel MLP conditioning (one-hot lang → 2048-dim hidden → 1024-dim encoder conditioning):

Code	Language	Code	Language	Code	Language
ar-AR	Arabic	fr-CA	French (CA)	nn-NO	Norwegian (NN)
bg-BG	Bulgarian	fr-FR	French (FR)	pl-PL	Polish
cs-CZ	Czech	he-IL	Hebrew	pt-BR	Portuguese (BR)
da-DK	Danish	hi-IN	Hindi	pt-PT	Portuguese (PT)
de-DE	German	hr-HR	Croatian	ro-RO	Romanian
el-GR	Greek	hu-HU	Hungarian	ru-RU	Russian
en-GB	English (GB)	it-IT	Italian	sk-SK	Slovak
en-US	English (US)	ja-JP	Japanese	sl-SI	Slovenian
es-ES	Spanish (ES)	ko-KR	Korean	sv-SE	Swedish
es-US	Spanish (US)	lt-LT	Lithuanian	th-TH	Thai
et-EE	Estonian	lv-LV	Latvian	tr-TR	Turkish
fi-FI	Finnish	nb-NO	Norwegian (NB)	uk-UA	Ukrainian
		nl-NL	Dutch	vi-VN	Vietnamese
				zh-CN / zh-TW	Chinese

Key properties:

Sample rate: 16 kHz mono
128 mel filterbank features, n_fft=512, hop=160 (10ms), win=400 (25ms)
8x time downsampling (causal) → 80ms frame duration
Streaming: cache-aware attention with 4 context presets (see below)
Vocab: 13087 SentencePiece tokens + 1 blank (pure RNN-T, no TDT durations)
Native punctuation and capitalization
License: OpenMDW-1.1 (permissive, commercial OK)

Files

File	Size	Description
`nemotron-3.5-asr-streaming-0.6b-f16.gguf`	1.3 GB	F16 weights (full precision, F32 pre-encode)
`nemotron-3.5-asr-streaming-0.6b-q4_k.gguf`	458 MB	Recommended. Q4_K quantized, identical text output, ~2x faster.

Pre-encode weights are kept at F32 in both GGUFs (F16 causes 1.56 max accumulation error across the 4352-dim projection).

Usage with CrispASR

# Auto-download (Q4_K, 458 MB)
crispasr --backend nemotron -m auto --auto-download -f audio.wav

# Or download manually
huggingface-cli download cstr/nemotron-3.5-asr-streaming-GGUF \
  nemotron-3.5-asr-streaming-0.6b-q4_k.gguf --local-dir models/

# Transcribe (English, default)
crispasr --backend nemotron \
  -m models/nemotron-3.5-asr-streaming-0.6b-q4_k.gguf \
  -f audio.wav

# Transcribe in German
crispasr --backend nemotron \
  -m models/nemotron-3.5-asr-streaming-0.6b-q4_k.gguf \
  -f audio.wav -l de-DE

# Beam search (default is greedy)
crispasr --backend nemotron -m auto --auto-download -f audio.wav --beam-size 4

# Streaming from stdin
ffmpeg -i audio.wav -f s16le -ar 16000 -ac 1 - | \
  crispasr --backend nemotron -m auto --auto-download --stream

Streaming encoder

The model supports true cache-aware streaming via the NeMo cache_last_channel + cache_last_time architecture. Enable with:

CRISPASR_NEMOTRON_STREAMING=1 crispasr --backend nemotron -m model.gguf -f audio.wav

Four attention context presets trade latency for accuracy (published WER from NVIDIA's Open ASR Leaderboard):

Preset	Left ctx	Right ctx	Chunk size	Approx latency	Published WER
0 (default)	56 frames	3 frames	4 frames	~160 ms	7.67%
1	56 frames	0 frames	1 frame	~80 ms	8.43%
2	56 frames	6 frames	7 frames	~560 ms	7.07%
3	56 frames	13 frames	14 frames	~1120 ms	6.93%

Select preset: CRISPASR_NEMOTRON_CONTEXT_PRESET=3 (default: 0)

The same GGUF works for all presets — the context window is a runtime knob, not a retraining artifact.

Environment variables

Variable	Effect
`CRISPASR_NEMOTRON_STREAMING=1`	Enable cache-aware streaming encoder
`CRISPASR_NEMOTRON_CONTEXT_PRESET=N`	Attention context preset (0-3)
`CRISPASR_NEMOTRON_NO_WINDOW_MASK=1`	Disable banded attention mask (bidirectional fallback)
`CRISPASR_NEMOTRON_DEBUG=1`	Enable encoder/decoder debug prints

Architecture

Audio (16kHz mono)
  → Mel spectrogram (128 bins, 10ms hop, no normalization)
  → Pre-encode (3x causal Conv2d, 8x downsample, Linear 4352→1024, F32 weights)
  → 24x Cache-Aware FastConformer block:
      FFN1(½) → MHA(rel_pos, cache-aware) → DWConv(k=9, causal, LN) → FFN2(½) → LN
  → Prompt kernel (MLP: concat(enc[1024], lang_onehot[128]) → 2048 → ReLU → 1024)
  → RNN-T decoder:
      Prediction: Embed(13088, 640) + 2-layer LSTM(640)
      Joint: enc(1024→640) + pred(640→640) → ReLU → Linear(640→13088)
  → Greedy / beam search decode

Streaming caches (per layer):

cache_last_channel: post-FFN1 output (up to L frames), used as K/V context for asymmetric attention (Q from new frames only)
cache_last_time: last K-1=8 frames of post-GLU signal before depthwise conv, prepended instead of zero-padding

Conversion

python models/convert-nemotron-to-gguf.py \
  --nemo nvidia/nemotron-3.5-asr-streaming-0.6b \
  --output nemotron-3.5-asr-streaming-0.6b-f16.gguf

crispasr-quantize nemotron-3.5-asr-streaming-0.6b-f16.gguf \
  nemotron-3.5-asr-streaming-0.6b-q4_k.gguf q4_k

Quality reference (JFK 11s)

Variant	Output
F16	And so my fellow Americans ask not what your country can do for you. <en-US> Ask what you can do for your country. <en-US>
Q4_K	And so my fellow Americans ask not what your country can do for you. <en-US> Ask what you can do for your country. <en-US>
Streaming (preset 0)	And so, my fellow Americans ask not what your country can do for you. <en-US> Ask what you can do for your country. <en-US>

F16 and Q4_K produce identical text. Streaming output has minor punctuation differences but same content.

Original model

Source: nvidia/nemotron-3.5-asr-streaming-0.6b
License: OpenMDW-1.1 — permissive, commercial use OK, derivatives OK with attribution
Training data: 530k hours (NVIDIA Riva ASR set + Granary)

0x3/nemotron-3.5-asr-streaming-0.6b-GGUF