Nemotron 3 Diarization GGUF for speech.cpp

These files run only in speech.cpp 0.9.0 or later. Their layout is speech.cpp's own, so llama.cpp, LM Studio, Ollama and other GGUF readers cannot run them.

Each holds NVIDIA's Nemotron 3 Diarization, a Streaming Sortformer of 99M parameters, with its mel frontend and the four configurations of its model card. It finds who speaks when, up to eight speakers at once, in a whole recording or in audio as it arrives, and numbers the speakers in the order they first speak. It takes any language.

Use

Install speech.cpp, then:

speech diarize nemotron-3-diarization meeting.wav                 # FILE, START, END and SPEAKER of each segment
speech diarize nemotron-3-diarization --format rttm a.wav b.wav > hyp.rttm
speech asr qwen3-asr-1.7b --vad silero-vad --diarize nemotron-3-diarization meeting.wav
speech serve qwen3-asr-1.7b silero-vad nemotron-3-diarization      # speakers in diarized_json and on /v1/realtime

The name nemotron-3-diarization fetches the Q8_0 file the first time it is used, and nemotron-3-diarization:f16 or :f32 the others; a path to a downloaded file works as well. A program calls the C API's speech_diarize(), or its diarizations of audio given a piece at a time. speech.cpp's documentation has the model's page, the command line and the server.

A whole recording is diarized in the card's offline configuration, which hears 30.4 s at a time, and audio as it arrives in low_latency, which waits for 1.04 s; very_low_latency (0.64 s) and ultra_low_latency (0.32 s) are the card's others, and a request may set any of them.

Files

Accuracy

Each stage is checked against NeMo at commit d32dd55 on three inputs, 109 s in all, of FLEURS clips of five voices with pauses, overlaps, silence and noise, in each configuration. The probabilities of F32 differ from the official by 3.2e-5 at most on an Apple M5's CPU, 0.12 on its GPU (Metal), where the speaker cache once keeps another frame, and 1.2e-5 on an NVIDIA GeForce RTX 2080 (Vulkan).

The diarization error rate on the test set of AMI (16 meetings, 9.1 h, the headset mix), against NTT's forced-alignment references with a collar of 0 s and overlapped speech included, as the model card scores it, with pyannote.metrics 3.2.1, on an Apple M5's GPU:

Speed

The RTFx (the seconds of audio over the seconds they took) of speech diarize on a 98 s recording, one request at a time:

A stream hears the speaker cache and the queue again with each chunk, so it takes about twenty times as long as offline; on a CPU it leaves little for a recognizer.

Source

Converted by speech.cpp's reference/sortformer/convert.py from Nemotron-3-Diarization.nemo (198,676,480 bytes, SHA-256 867c53f552998f772e5b5e5c082962ae85ee7ca5669c2bc17d7f615133d4e96d) and the streaming configurations of config.json and processor_config.json of nvidia/Nemotron-3-Diarization at revision f667ed73aee57d40cc39428eb768b4fd87a0a29e, with NeMo at commit d32dd550ffb016cd7b4133bb2bc69f15ba866361. The two heads that no inference reads, hidden_to_spks and the activity head of training, are left out. The F16 and Q8_0 files are made from the F32 one with speech quantize --type F16 and --type Q8_0.

License

Nemotron 3 Diarization is NVIDIA's, released under the OpenMDW License Agreement, version 1.1, which LICENSE.OpenMDW-1.1 holds as github.com/OpenMDW/openmdw has it at commit b26b32b34ad2edcc29a7707abb68dcfb25a538c1. These files are distributed under that agreement. NVIDIA's model card has the model's ethical considerations and its subcards on bias, explainability, safety and privacy.