Chatterbox Multilingual V3 — BF16

BF16 conversion of mlx-community/chatterbox-multilingual-v3, retaining its multilingual V3 architecture and tokenizer. This is a precision conversion, not a new fine-tune.

Source revision: 03565773edd72e949572557597af8063bb49a18a. All 2,487 source tensors (677,696,431 floating parameters) were converted from FP32 to BF16. Shapes and dtypes were verified after saving.

Weight bytes: 2,711,113,780 → 1,355,722,630. BF16 model.safetensors SHA-256: cec001c644dce73725b4091f3310741eb7af5764d88f985188fb47a05046c1a6.

Multilingual V3 runtime compatibility

This checkpoint uses the Multilingual V3 speech architecture. For MLX-Audio builds whose HiFT V3 decoder applies a final LeakyReLU negative slope of 0.1 before conv_post, the official V3 source uses 0.01 at that location. Check the runtime implementation or use a version with the V3 decoder corrected; the intermediate slopes remain unchanged. This is a decoder compatibility consideration and does not require re-quantizing the model.

The LatAm regular Q8 release contains a source-guarded example and measured saturation checks. Those LatAm results do not certify this checkpoint; retain the model-specific evaluations and validate audio from your own runtime.

Runtime

Tested with MLX 0.32.3 and mlx-audio 0.5.9 on Ubuntu 24.04 / RTX 3060 12 GB. MLX also supports Apple silicon; the measurements below apply to the tested NVIDIA setup.

The S3TokenizerV2 dependency downloads separately. Its weights remain FP32. The included infer.py casts the T3 transformer input embeddings to BF16; the stock upstream conditioning path can otherwise promote them back to FP32. Vocoder intermediate computation can remain mixed precision.

On Linux / CUDA:

pip install 'mlx[cuda12]==0.32.3' 'mlx-audio==0.5.9' soundfile
export MLX_CUDA_CONV_CACHE_SIZE=2048
export MLX_CUDA_GRAPH_CACHE_SIZE=2048

On Apple silicon, install mlx==0.32.3 instead of the CUDA extra.

Download this repository to a local folder, then run:

python infer.py --reference your_voice.wav --language es \
  --text 'Hola. Esta es una prueba de voz en español.' --output speech.wav

The reference is supplied by the user. No personal reference recordings or precomputed voice conditionals are distributed here. Use --no-t3-cast to reproduce the stock BF16 input path.

RTX 3060 measurements

Nine timed Spanish/English samples per profile after initial and per-shape warmups; paired seeds 101/102/103. Generation time includes complete GPU waveform evaluation and host copy, excludes model loading, reference preparation and file encoding. RTF below one is faster than real time.

Peak MLX allocation is not total GPU VRAM. Warm performance is not a guarantee for arbitrary prompt shapes. The implementation yields a complete utterance and does not provide native streaming audio. No ASR word-error or objective speaker-similarity evaluation was performed for these original measurements.

Original architecture/model: ResembleAI Chatterbox. MLX conversion/runtime: mlx-community / mlx-audio contributors. License: MIT, preserving upstream attribution. See LICENSE and provenance.json.