Chatterbox Nano — CoreML

CoreML export of ResembleAI/chatterbox-nano (110M GPT2-small T3 + distilled 2-step meanflow S3Gen, English, paralinguistic tags) for Apple platforms, converted by FluidInference.

Each model ships as both .mlpackage (source) and compiled .mlmodelc.

Models

Output budget / bucket choice

The flow bucket holds voice prompt tokens + generated speech tokens + 3 silence tokens. The built-in voice uses 250 prompt tokens, so N500 yields ≤247 generated tokens ≈ 9.9 s of audio per call and N1000 ≤747 ≈ 29.9 s. Likewise the T3 prefill window holds voice conditioning (376) + text BPE + 1 BOS, i.e. ≤135 text tokens. Pick ONE flow/vocoder pair at load time; the buckets are static shapes, so the extended pair roughly doubles flow+vocoder latency even for short outputs.

The AR stage runs at ~300 tok/s against the 25 Hz speech-token rate (~12× real-time) with the stateful decode; the full compute path is ~3.4× real-time with I/O-KV decode and ~6× with MLState, measured on an M-series Mac with .cpuAndGPU.

⚠️ The sibling Multilingual T3 packages hard-crash under .cpuOnly; the Nano packages have not been matrix-tested there — prefer .cpuAndGPU or .all.

Samples

samples/ has CoreML end-to-end renders (e2e_nano_*.wav) next to stock PyTorch renders (baseline_*.wav), all using the built-in voice. *_tags.wav exercises the [chuckle] paralinguistic tag. Parakeet-v3 transcribes both e2e renders verbatim.

To synthesize locally without the upstream checkpoint (Apple silicon):

git clone -b feat/chatterbox-nano-coreml https://github.com/FluidInference/mobius
cd mobius/models/tts/chatterbox/coreml
uv sync
uv run python verify/e2e_nano_coreml.py   # models auto-download from this repo

Runtime boundary

The graphs cover T3 prefill/decode, the meanflow flow, and the HiFT vocoder. The host runtime must provide:

• text normalization (punc_norm) + BPE tokenization (tokenizer/; no BOT/EOT wrapping — the prefill context is cond ++ text ++ one BOS speech embed)
• embedding prep from tables.safetensors (no positional add — GPT2 wpe is applied in-graph)
• turbo sampling: temperature 0.8 → top-k 1000 → top-p 0.95 → repetition penalty 1.2, EOS = 6562 (no CFG, no alignment analyzer)
• SineGen randomness (phase_vec, noise inputs to HiFT) and flow noise z
• flow bucket padding/cropping; MLState seeding from prefill KV for the stateful decode

Voice cloning from a reference wav additionally needs the VoiceEncoder / S3TokenizerV2 / CAMPPlus encoders, which are not converted here; voices can be prepared offline in Python (export-tables-nano.py --ref-wav) and shipped as voice-*.safetensors.

Parity (vs upstream PyTorch)

License

MIT, following upstream ResembleAI/chatterbox-nano. Upstream embeds Resemble's Perth watermarker in its Python pipeline; this CoreML export does not include a watermarking stage.