中文Sign up

FluidInference/chatterbox-nano-coreml

🤗 Hugging Face sourcetext-to-speechmit1.0 GBother✓ 42 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo FluidInference/chatterbox-nano-coreml ./model-folder
Needs a seeder →

Chatterbox Nano — CoreML

CoreML export of ResembleAI/chatterbox-nano (110M GPT2-small T3 + distilled 2-step meanflow S3Gen, English, paralinguistic tags) for Apple platforms, converted by FluidInference.

Each model ships as both .mlpackage (source) and compiled .mlmodelc.

Models

File Size (fp16) Role Compute
T3Nano-Prefill-T512-M1536-fp16 173 MB GPT2-small prefill over ≤512-token context (batch 1, no CFG), initializes 1536-slot KV cache CPU+GPU
T3Nano-Decode-M1536-fp16 184 MB Single-step AR decode, KV cache via I/O tensors (9 ms/step) CPU+GPU
T3Nano-Decode-M1536-fp16-stateful 184 MB Single-step AR decode, KV cache in MLState (3.3 ms/step; macOS 15+/iOS 18+) CPU+GPU
FlowMean-N500-fp16 228 MB S3Gen meanflow: 500-token bucket → 1000 mel frames, 2 plain Euler steps in-graph (0.38 s/call) CPU+GPU
HiFT-T1000-fp16 40 MB HiFTNet vocoder: mel → 24 kHz waveform (0.09 s/call) CPU+GPU
FlowMean-N1000-fp16 242 MB S3Gen meanflow, 1000-token bucket → 2000 mel frames (extended ~30 s output; FluidAudio #924) CPU+GPU
HiFT-T2000-fp16 40 MB HiFTNet vocoder for the 2000-frame extended bucket CPU+GPU
tables/tables.safetensors 87 MB text/speech embedding tables (host applies; GPT2 wpe is in-graph)
tables/voice-default.safetensors 0.7 MB precomputed built-in voice conditioning (T3 cond embeds + S3Gen ref dict)
tokenizer/ 1.4 MB GPT2 BPE tokenizer (50276 ids incl. 20 paralinguistic tags: [laugh], [chuckle], [sigh], …)

Output budget / bucket choice

The flow bucket holds voice prompt tokens + generated speech tokens + 3 silence tokens. The built-in voice uses 250 prompt tokens, so N500 yields ≤247 generated tokens ≈ 9.9 s of audio per call and N1000 ≤747 ≈ 29.9 s. Likewise the T3 prefill window holds voice conditioning (376) + text BPE + 1 BOS, i.e. ≤135 text tokens. Pick ONE flow/vocoder pair at load time; the buckets are static shapes, so the extended pair roughly doubles flow+vocoder latency even for short outputs.

The AR stage runs at ~300 tok/s against the 25 Hz speech-token rate (~12× real-time) with the stateful decode; the full compute path is ~3.4× real-time with I/O-KV decode and ~6× with MLState, measured on an M-series Mac with .cpuAndGPU.

⚠️ The sibling Multilingual T3 packages hard-crash under .cpuOnly; the Nano packages have not been matrix-tested there — prefer .cpuAndGPU or .all.

Samples

samples/ has CoreML end-to-end renders (e2e_nano_*.wav) next to stock PyTorch renders (baseline_*.wav), all using the built-in voice. *_tags.wav exercises the [chuckle] paralinguistic tag. Parakeet-v3 transcribes both e2e renders verbatim.

To synthesize locally without the upstream checkpoint (Apple silicon):

git clone -b feat/chatterbox-nano-coreml https://github.com/FluidInference/mobius
cd mobius/models/tts/chatterbox/coreml
uv sync
uv run python verify/e2e_nano_coreml.py   # models auto-download from this repo

Runtime boundary

The graphs cover T3 prefill/decode, the meanflow flow, and the HiFT vocoder. The host runtime must provide:

  • text normalization (punc_norm) + BPE tokenization (tokenizer/; no BOT/EOT wrapping — the prefill context is cond ++ text ++ one BOS speech embed)
  • embedding prep from tables.safetensors (no positional add — GPT2 wpe is applied in-graph)
  • turbo sampling: temperature 0.8 → top-k 1000 → top-p 0.95 → repetition penalty 1.2, EOS = 6562 (no CFG, no alignment analyzer)
  • SineGen randomness (phase_vec, noise inputs to HiFT) and flow noise z
  • flow bucket padding/cropping; MLState seeding from prefill KV for the stateful decode

Voice cloning from a reference wav additionally needs the VoiceEncoder / S3TokenizerV2 / CAMPPlus encoders, which are not converted here; voices can be prepared offline in Python (export-tables-nano.py --ref-wav) and shipped as voice-*.safetensors.

Parity (vs upstream PyTorch)

Check Result
T3 wrappers vs stock (fp32) logits 1.4e-05
T3 CoreML fp16 (I/O-KV / stateful) logits 2.2e-02 / 1.9e-02
Flow CoreML fp16 mel max 5.9e-02, mean 1.8e-03
HiFT CoreML fp16 wav max 3.3e-03, mean 1.2e-04
e2e ASR round-trip verbatim transcripts; [chuckle] audibly realized

License

MIT, following upstream ResembleAI/chatterbox-nano. Upstream embeds Resemble's Perth watermarker in its Python pipeline; this CoreML export does not include a watermarking stage.