VibeVoice-Realtime-0.5B as GGUF for livevoice.cpp, a C++/ggml runtime for streaming text-to-speech: it starts speaking while the text is still being written, about 70-120 ms after the first words on an M4 Max.
Each file is self-contained: the model weights, the tokenizer and a DeepFilterNet3 post-filter that removes the short musical artefact the model sometimes puts in front of a reply. Output is 48 kHz mono.
| file | size | contents |
|---|---|---|
livevoice-0.5b-compact.gguf |
577 MB | q8_0 attention and diffusion head; iq4_nl gate / up; q4_k ffn_down; q4_0 embedding without the rows the tokenizer never produces; q5_k decoder FFN; q8_0 decoder convs. Recommended: smallest, 0.9 GB of memory, same speed and pronunciation as attn-q8. Needs livevoice.cpp from 2026-11-10 or later |
livevoice-0.5b-attn-q8.gguf |
757 MB | q8_0 attention, token embedding and diffusion head; q4_0 gate / up; q4_k ffn_down; q6_k decoder FFN. The previous default |
livevoice-0.5b-mixed.gguf |
862 MB | attn-q8 with q8_0 gate / up; no measurable gain, slower on phones |
livevoice-0.5b-q4_k.gguf |
667 MB | q4_0 attention, gate / up and embedding; q4_k ffn_down; q6_k decoder FFN. Worse on rare words |
livevoice-0.5b-q8_0.gguf |
1.1 GB | q8_0 throughout, F16 embedding |
voices/en-Clarissa_female.gguf |
7 MB | voice prompt |
voices/en-Clarion_male.gguf |
7 MB | voice prompt |
The realtime model has no audio encoder, so a voice is a pre-computed prompt (the model's K/V caches after hearing the voice) rather than a reference recording. Any of the voices above works with any of the model files.
git clone --recursive https://github.com/nodcai/livevoice.cpp && cd livevoice.cpp
cmake -B build && cmake --build build -j
hf download nodcai/livevoice-0.5b-GGUF --local-dir models
./build/bin/livevoice-cli --model models/livevoice-0.5b-compact.gguf \
--voice models/voices/en-Clarissa_female.gguf \
--text "Hello! This is a streaming voice." --stream --out hello.wav
From C, push text as an LLM produces it and receive audio chunks as they are generated:
lv_model* model = lv_model_load("livevoice-0.5b-compact.gguf");
lv_voice* voice = lv_voice_load(model, "voices/en-Clarissa_female.gguf");
lv_params p;
lv_params_init(&p);
lv_stream* s = lv_stream_begin(model, voice, &p, on_audio, user);
lv_stream_push(s, "Hello th");
lv_stream_push(s, "ere! How can I help?");
lv_stream_end(s);
lv_stream_free(s);
Defaults are 5 diffusion steps and a guidance scale of 1.5. See the livevoice.cpp README for the C API and every parameter.
K-quants pack 256 weights per block and the LM's hidden size is 896, so
only ffn_down can take q4_k; every other LM matrix and the embedding fall
back to a block-32 type. With q4_0 there, rare words suffer. Rare-word
pronunciation, 24 sentences with 35 rare-word slots x 10 seeds, judged by an
ASR model:
| LM attention / gate / up | rare-word slots hit |
|---|---|
| q4_0 (the q4_k file) | 73.7 % |
| q8_0 | 79.7 % |
| f16 | 78.3 % |
The attention projections carry the gain and are only 47 MB at q8_0. Every LM matrix is read once per speech frame, so on a phone GPU memory bandwidth sets the speed: on an iPhone 18 Pro (Metal, 5 steps, CFG 1.5) q4_k runs at 8.0x real time, attn-q8 at 7.9x, q8_0 gate / up at 6.3x. With livevoice.cpp's tokenizer rules for rare words (on by default), attn-q8 hits 82.0 % of the slots. Everyday-word error rates are the same for all of these files.
compact spends the bytes where the model is sensitive. Given the same text and the same speech latents, the language model's output drifts from the q8_0 file's by 0.7 % with the token embedding at q4_0, but by about 10 % with either the attention or the gate / up matrices at q4_0. So it keeps attention at q8_0, stores gate / up as iq4_nl (8.7 % drift, same size as q4_0), and takes the embedding down to q4_0 after dropping the 69,908 rows the tokenizer cannot produce under livevoice.cpp's merge cap of 80000. The decoder's upsampling kernels go from F16 to q8_0 and its FFN from q6_k to q5_k; the diffusion head, the most sensitive part, stays at q8_0.
| attn-q8 | compact | |
|---|---|---|
| file | 757 MB | 577 MB |
| language-model drift from the q8_0 file | 13.0 % | 11.7 % |
| decoder waveform difference from the q8_0 file | 1.9 % | 3.4 % |
| rare words transcribed correctly (990 slots, Parakeet as judge) | 94.7 % | 94.6 % |
| final word of a sentence correct (400 endings) | 96.2 % | 96.5 % |
| peak memory, M4 Max | 1.50 GB | 0.92 GB |
| time to 10.4 s of audio, M4 Max | 0.93 s | 0.94 s |
Two things to know: a compact file limits merge_cap to 80000 (the
default), and on one test sentence it says "cryptocurrencies" for a final
"cryptocurrency" more often than attn-q8 does (12 of 30 against 1 of 30).
Speed on phone GPUs has not been measured yet.
python scripts/convert_tokenizer.py --src Qwen2.5-0.5B --out tokenizer.gguf
python scripts/convert_deepfilternet.py --input DeepFilterNet3 --output deepfilternet3.gguf
python scripts/convert_model.py --src VibeVoice-Realtime-0.5B \
--tokenizer tokenizer.gguf --dfn deepfilternet3.gguf --out livevoice-0.5b-f16.gguf
livevoice-quantize --src livevoice-0.5b-f16.gguf --out livevoice-0.5b-compact.gguf --recipe compact
The other files use the recipes attn-q8, mixed, q4_k and q8_0. Voice prompts
in the upstream .pt format (such as those in demo/voices/streaming_model
of microsoft/VibeVoice) convert
with scripts/convert_voice.py.
MIT, as the base model microsoft/VibeVoice-Realtime-0.5B; see its card for the intended uses and limitations of the model, which apply here unchanged. The embedded DeepFilterNet3 weights are MIT / Apache-2.0.