Nemotron-3.5 ASR Streaming 0.6B — CoreML INT8

Cache-aware streaming Conformer + RNN-T from NVIDIA, ported to CoreML (.mlmodelc) for on-device inference on Apple Silicon. 600 M params, 40 language-locales, native punctuation and capitalization. INT8 palettized encoder, FP16 decoder + joint. Ships .mlmodelc (compiled) — .mlpackage is not shipped because on-device MLModel.compileModel() produces non-deterministic output across iOS simulator vs device runtimes.

Model

Files

Performance

M5 Pro (Apple Silicon), 50 samples per language from FLEURS test, streaming 320 ms chunks, compute units .all. Scoring uses Whisper EnglishTextNormalizer for English and BasicTextNormalizer(split_letters=True) for hi/ja (char-level), BasicTextNormalizer for de/fr/ar.

Accuracy

Quantization is essentially lossless. For ja/hi the published WER is char-level (matches NVIDIA's CJK methodology); CER is the more interpretable number for those scripts.

Streaming throughput + memory

M5 Pro, 60 s long-form en_us audio, single thread, .all compute units:

Usage

Python (coremltools)

import coremltools as ct
import numpy as np
import json
from huggingface_hub import snapshot_download

bundle = snapshot_download("aufklarer/Nemotron-3.5-ASR-Streaming-0.6B-CoreML-INT8")
encoder = ct.models.CompiledMLModel(f"{bundle}/encoder.mlmodelc")
decoder = ct.models.CompiledMLModel(f"{bundle}/decoder.mlmodelc")
joint   = ct.models.CompiledMLModel(f"{bundle}/joint.mlmodelc")

slots = json.load(open(f"{bundle}/languages.json"))["promptDictionary"]
lang_mask = np.zeros((1, 128), dtype=np.float32)
lang_mask[0, slots["en-US"]] = 1.0
# Feed 320 ms chunks; persist caches across calls (see inference reference below).

Swift (speech-swift SDK)

import NemotronStreamingASR

// Default model id is this repo; pass `modelId:` to pin
let model = try await NemotronStreamingASRModel.fromPretrained()

// Batch
let text = try model.transcribeAudio(audio, sampleRate: 16000, language: "en-US")

// Streaming (yields partials as audio is fed)
for await partial in model.transcribeStream(audio: audio, sampleRate: 16000, language: "ja-JP") {
    print(partial.text, partial.isFinal)
}

Language is a BCP-47 tag (en-US, de-DE, fr-FR, ja-JP, hi-IN, ...) — resolved via languages.json. The Swift wrapper persists the streaming KV/conv caches across pushAudio calls within a StreamingSession.

CLI (Homebrew)

brew install soniqo/tap/speech

# Single-shot transcription on a wav file (uses this CoreML INT8 bundle by default)
speech transcribe recording.wav --engine nemotron --language en-US

# Other languages
speech transcribe meeting.wav --engine nemotron --language de-DE
speech transcribe interview.wav --engine nemotron --language ja-JP

speech is a universal binary; the model is downloaded on first use and cached under ~/Library/Caches/qwen3-speech/.

Source

Upstream: nvidia/nemotron-3.5-asr-streaming-0.6b.

License

This CoreML conversion is distributed under OpenMDW 1.1, matching the upstream NVIDIA model. NVIDIA marks the upstream model ready for commercial use.

Links

• speech-swift — Apple SDK with NemotronStreamingASR
• Docs — install and CLI documentation
• soniqo.audio
• blog