中文Sign up

aufklarer/Chatterbox-Flash-CoreML

🤗 Hugging Face sourcetext-to-speechmit2.4 GBother✓ 22 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo aufklarer/Chatterbox-Flash-CoreML ./model-folder
Needs a seeder →

Chatterbox Flash CoreML

Compiled Core ML export of ResembleAI/chatterbox-flash for Apple runtimes.

This bundle contains the Chatterbox Flash T3 token generator and the S3Gen audio back-half as static-shape .mlmodelc graphs. It is intended for an application runtime that owns text tokenization, T3 denoising/sampling, reference conditioning, and graph orchestration.

Links

Model

Component Parameters Format Precision Static shape
T3 block-diffusion token generator 532.4M Core ML .mlmodelc fp16 text_len 256, block_size 16, max_seq 1024
S3Gen audio back-half 266.0M Core ML .mlmodelc fp16 token_len 192, mel_len 384
Total 798.4M Core ML .mlmodelc fp16 24 kHz waveform output

Files

Path Size Description
config.json 4 KB Root metadata for download tracking and runtime discovery
t3/ConditioningEncoder.mlmodelc 25 MB Speaker/prompt/emotion conditioning to T3 conditioning embedding
t3/TextPrefill.mlmodelc 963 MB Causal [cond, text, start_speech] prefix prefill and flat KV cache
t3/BlockDecoder.mlmodelc 1.0 GB Full-visible Flash speech-block logits with explicit KV cache
t3/uncond_block_prior.npy 36 KB Unconditional PMI prior for Flash scoring
t3/tokenizer.json 28 KB Chatterbox Flash text tokenizer
t3/config.json 4 KB T3 export metadata
audio/FlowSpeakerProjector.mlmodelc 44 KB S3Gen reference embedding to projected speaker conditioning
audio/FlowEncoder.mlmodelc 79 MB prompt_token ++ speech_tokens to flow mu and mask
audio/FlowEstimator.mlmodelc 141 MB One meanflow Euler derivative step
audio/HiFTVocoder.mlmodelc 41 MB Mel frames to 24 kHz waveform
audio/audio_config.json 4 KB S3Gen audio export metadata

Runtime Boundary

The exported graphs cover:

  • T3 conditioning, text prefill, and block decoding.
  • S3Gen speaker projection, flow encoder, meanflow estimator step, and HiFT vocoder.

The host runtime must still provide:

  • text normalization and tokenization
  • T3 denoising loop, PMI/CFG scoring, sampling, and EOS trimming
  • reference waveform encoders:
    • VoiceEncoder: ref.wav -> speaker_emb
    • prompt speech tokenizer: ref.wav -> prompt_speech_tokens
    • S3Gen reference encoder: ref.wav -> prompt_token, prompt_feat, embedding
  • padding/cropping and the two-step meanflow loop over (t,r) = (0,0.5), (0.5,1.0)

This means the bundle supports voice-cloning TTS when the runtime supplies reference conditioning tensors, but it is not yet a fully Core ML ref.wav -> cloned wav pipeline.

Validation

Test Result
T3 graph roundtrip vs PyTorch wrappers Pass at 2% relative tolerance
Audio graph roundtrip vs PyTorch wrappers Pass for token_len 192, mel_len 384
Stitched S3Gen meanflow-to-audio roundtrip Pass
Prompted synthesis smoke test Pass
Whisper tiny transcript of generated wav Core ML speech test.
Smoke-test WER 0.000

Core ML warnings from local export:

  • CPU_ONLY prediction crashed for the T3 package in local coremltools 8.3 testing. Use ALL, CPU_AND_NE, or a compiled-device runtime.
  • The uploaded artifact ships compiled .mlmodelc folders. The numerical parity tests were run against the source .mlpackage exports before compilation.

Usage Sketch

The runtime loads graphs from t3/ and audio/, then:

  1. Prepare T3 reference conditioning tensors and S3Gen ref_dict tensors from a prompt wav.
  2. Tokenize text with t3/tokenizer.json.
  3. Run T3 prefill and block denoising/sampling to produce S3 speech tokens.
  4. Concatenate S3Gen prompt_token and generated speech tokens.
  5. Run audio/FlowEncoder.mlmodelc.
  6. Build cond by copying prompt_feat.T into the mel prefix.
  7. Run audio/FlowEstimator.mlmodelc twice for (0,0.5) and (0.5,1.0).
  8. Crop generated mel frames after the prompt prefix.
  9. Run audio/HiFTVocoder.mlmodelc and crop padded samples.

Source

Converted from ResembleAI/chatterbox-flash, revision 4385507288b8197e6dab8b4e6b1603328d549d9d.