中文Sign up

FluidInference/chatterbox-multilingual-coreml

🤗 Hugging Face sourcetext-to-speechmit2.4 GBother✓ 33 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo FluidInference/chatterbox-multilingual-coreml ./model-folder
Needs a seeder →

Chatterbox Multilingual — CoreML

CoreML export of ResembleAI/chatterbox multilingual (23 languages, t3_mtl23ls_v2 + s3gen) for Apple platforms, converted by FluidInference (conversion toolkit: mobius PR #89).

Each model ships as both .mlpackage (source) and compiled .mlmodelc.

Models

File Size (fp16) Role Compute
T3-Prefill-T256-M1024-fp16 977 MB Llama-520M prefill over ≤256-token context (CFG batch 2), initializes 1024-slot KV cache CPU+GPU
T3-Decode-M1024-fp16 977 MB Single-step AR decode, KV cache via I/O tensors (38 ms/step) CPU+GPU
T3-Decode-M1024-fp16-stateful 977 MB Single-step AR decode, KV cache in MLState (16.8 ms/step; macOS 15+/iOS 18+) CPU+GPU
Flow-N500-fp16 229 MB S3Gen flow: 500-token bucket → 1000 mel frames, 10-step CFG Euler in-graph CPU+GPU
HiFT-T1000-fp16 40 MB HiFTNet vocoder: mel → 24 kHz waveform (0.09 s/call) CPU+GPU
tables/tables.safetensors 34 MB text/speech embedding + learned positional tables (host applies)
tables/voice-default.safetensors 0.1 MB precomputed built-in voice conditioning (T3 cond embeds + S3Gen ref dict)
tokenizer/grapheme_mtl_merged_expanded_v1.json 23-language grapheme tokenizer

⚠️ Do not load the T3 packages with .cpuOnly — prediction hard-crashes (also independently reported by other Chatterbox CoreML ports). Use .cpuAndGPU or .all.

Samples

samples/ has CoreML end-to-end renders (e2e_*.wav) next to stock PyTorch renders (baseline_*.wav) for en/de/fr, all using the built-in voice.

To synthesize locally without the upstream checkpoint (Apple silicon):

git clone -b feat/chatterbox-mtl-coreml https://github.com/FluidInference/mobius
cd mobius/models/tts/chatterbox/coreml
uv sync
uv run python verify/e2e_coreml.py --lang en   # models auto-download from this repo

Runtime boundary

The graphs cover T3 prefill/decode (with the multilingual alignment-analyzer attention rows as outputs), the S3Gen flow, and the HiFT vocoder. The host runtime must provide:

  • text normalization + tokenization (tokenizer/)
  • embedding prep from tables.safetensors (text/speech + positional; the stock prefill context ends with two BOS embeds — replicate exactly)
  • CFG combine cond + w*(cond-uncond), repetition penalty, min-p/top-p, sampling, EOS handling
  • the AlignmentStreamAnalyzer heuristics, fed by the exported align_attn rows (reference port: verify/analyzer_port.py in the conversion toolkit)
  • SineGen randomness (phase_vec, noise inputs to HiFT) and CFM noise z
  • flow bucket padding/cropping; MLState seeding from prefill KV for the stateful decode

Voice cloning from a reference wav additionally needs the VoiceEncoder / S3TokenizerV2 / CAMPPlus encoders, which are not converted here; voices can be prepared offline in Python (export-tables.py --ref-wav) and shipped as voice-*.safetensors.

Parity (vs upstream PyTorch)

Check Result
T3 wrappers vs stock (fp32) logits 3.8e-05, alignment rows exact
T3 CoreML fp16 logits 2.6e-02 (range ±15), align 1.4e-03
Flow CoreML fp16 mel max 2.4e-02, mean 2.7e-03
HiFT CoreML fp16 wav max 1.7e-02, mean 2.6e-04
e2e ASR round-trip (en) exact transcript, matches PyTorch baseline

License

MIT, following upstream ResembleAI/chatterbox. Upstream embeds Resemble's Perth watermarker in its Python pipeline; this CoreML export does not include a watermarking stage.