Chatterbox TTS — GGUF (ggml-quantised)

GGUF / ggml conversion of ResembleAI/chatterbox for use with CrispStrobe/CrispASR.

Chatterbox is a full TTS pipeline: character tokenizer → T3 (30-layer Llama AR, 520M) → speech tokens → S3Gen (Conformer encoder + UNet1D CFM denoiser, 10 Euler steps) → HiFTGenerator vocoder (conv chains + Snake activations + iSTFT) → 24 kHz WAV. Distributed under MIT license.

This is the multilingual Chatterbox (23 languages: Arabic, Danish, German, Greek, English, Spanish, Finnish, French, Hebrew, Hindi, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese, Russian, Swedish, Swahili, Turkish, Chinese) — the T3 GGUF carries the 2454-token multilingual text tokenizer. Select the language with -l <code> (e.g. -l de, -l ja); English is the default.

Two GGUF files are needed: the T3 model (text → speech tokens) and the S3Gen model (speech tokens → audio).

Files

Note: vocoder weights (conv_pre, resblocks, conv_post, source fusion) are kept at F32 in all quant levels for audio quality. Quantization applies to the Conformer encoder, UNet decoder, and T3 Llama layers.

Quick start

# 1. Build CrispASR
git clone https://github.com/CrispStrobe/CrispASR
cd CrispASR
cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF
cmake --build build -j --target chatterbox

# 2. Pull both model files
huggingface-cli download cstr/chatterbox-GGUF chatterbox-t3-q8_0.gguf --local-dir .
huggingface-cli download cstr/chatterbox-GGUF chatterbox-s3gen-q8_0.gguf --local-dir .

# 3. Synthesise with the built-in default voice
./build/bin/crispasr --backend chatterbox \
    -m chatterbox-t3-q8_0.gguf \
    --codec-model chatterbox-s3gen-q8_0.gguf \
    --tts "Hello there, this is chatterbox speaking." \
    --tts-output out.wav

# 4. (Optional) clone a different speaker — bake a small voice GGUF
# from a reference WAV, then pass it via --voice. Requires the upstream
# python pkg: pip install chatterbox-tts
python models/bake-chatterbox-voice-from-wav.py \
    --input /path/to/reference.wav \
    --output my_voice.gguf

./build/bin/crispasr --backend chatterbox \
    -m chatterbox-t3-q8_0.gguf \
    --codec-model chatterbox-s3gen-q8_0.gguf \
    --voice my_voice.gguf \
    --tts "Cloned voice synthesising arbitrary text." \
    --tts-output cloned.wav

See docs/tts.md for the full Chatterbox voice-clone reference, including the per-call cache used by --server mode.

Architecture

Text → Character tokenizer (704 tokens)
     → T3 Llama AR (30 layers, 1024D, 16 heads, RoPE, SwiGLU, CFG)
     → 25 Hz speech tokens (6561 codebook)
     → Conformer encoder (6 pre + 4 post upsample, 512D, 8 heads)
     → 80-channel mel spectrogram
     → UNet1D CFM denoiser (1 down + 12 mid + 1 up, 256 ch, 10 Euler steps)
     → HiFTGenerator vocoder (3× ConvTranspose1d + 9 ResBlocks + Snake + iSTFT)
     → 24 kHz mono WAV

Quality verification

ASR roundtrip on Python reference mel (no source fusion, deterministic):

All quantization levels (F16/Q8_0/Q4_K) produce ASR-identical output on the reference mel.

Conversion

python models/convert-chatterbox-to-gguf.py \
    --input ResembleAI/chatterbox \
    --output-dir .

Requires pip install gguf safetensors torch huggingface_hub.

Related models

• cstr/lahgtna-chatterbox-v1-GGUF — Arabic T3 variant (MIT, shares S3Gen)
• cstr/orpheus-3b-0.1-ft-GGUF — Llama-3.2 + SNAC TTS
• cstr/qwen3-tts-0.6b-customvoice-GGUF — Qwen3-TTS with fixed speakers

License

MIT — same as the upstream ResembleAI/chatterbox.

Provenance and EU AI Act Art. 53 note

• Upstream model: ResembleAI/chatterbox — published by ResembleAI.
• Upstream licence: mit. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.