CosyVoice3-0.5B MLX 8-bit

CosyVoice 3 text-to-speech model converted to MLX safetensors format for Apple Silicon inference. This bundle uses 8-bit LLM quantization and keeps the DiT flow decoder in bf16 for better speech quality than the older low-bit DiT export.

Converted from FunAudioLLM/Fun-CosyVoice3-0.5B-2512.

Model Summary

Field	Value
Parameters	0.5B LLM + DiT flow decoder + HiFi-GAN vocoder
Quantization	LLM int8, group_size=64; DiT bf16; vocoder fp32
Format	MLX safetensors
Sample rate	24 kHz
Languages	Chinese, English, Japanese, Korean, German, Spanish, French, Italian, Russian
Voice cloning	Zero-shot reference conditioning via S3-Tokenizer-v3 and reference transcript

Components

Component	Architecture	Precision	File
LLM	Qwen2.5-0.5B, 24 layers, 896 hidden, 14Q/2KV heads	int8 group quantized	`llm.safetensors`
DiT Flow Matching	22-layer DiT, 1024 hidden, 16 heads, 10 ODE steps	bf16	`flow.safetensors`
HiFi-GAN Vocoder	NSF + F0 predictor + ISTFT	fp32	`hifigan.safetensors`
S3-Tokenizer-v3	Reference-audio encoder for zero-shot cloning	bf16	`speech_tokenizer.safetensors`
Tokenizer	Qwen2.5 BPE	JSON/BPE	`vocab.json`, `merges.txt`, `tokenizer_config.json`

Files

File	Size	Description
`llm.safetensors`	640 MiB	8-bit quantized speech-token LLM
`flow.safetensors`	634 MiB	bf16 DiT flow decoder
`hifigan.safetensors`	79 MiB	fp32 vocoder with folded weight norm
`speech_tokenizer.safetensors`	462 MiB	bf16 S3-Tokenizer-v3 reference encoder
`config.json`	2 KiB	Runtime architecture and quantization metadata
`vocab.json`	2.6 MiB	Qwen2.5 BPE vocabulary
`merges.txt`	1.3 MiB	Qwen2.5 BPE merges
`tokenizer_config.json`	1 KiB	Tokenizer special-token metadata

Performance

Measured with speech-swift debug CLI on Apple Silicon. Numbers are sanity checks for runtime compatibility, not a final benchmark suite.

Test	Result
Load path	`Bundle quantization (LLM): 8-bit (group_size 64)`
English synthesis smoke	6.16 s audio in 2.44 s generation time
Real-time factor	0.40 RTF
ASR roundtrip	`This fixed eight-bit cozy voice export still loads.`

Pipeline

Text          ─┐
                ├─► LLM (Qwen2.5-0.5B int8) ─► Speech tokens (FSQ 6561)
Ref transcript ┘                                          │
                                                          ▼
                              ┌─► prompt_token ─┐
Reference WAV ─► S3-Tokenizer-v3                ├─► DiT Flow Matching ─► Mel
              ─► Matcha mel    ─► prompt_feat ─┘    (bf16)                  │
              ─► CAM++         ─► flow_embedding                             ▼
                                                                         HiFi-GAN
                                                                             │
                                                                             ▼
                                                                        Audio (24 kHz)

Usage

Swift

import CosyVoiceTTS

let model = try await CosyVoiceTTSModel.fromPretrained(
    modelId: "aufklarer/CosyVoice3-0.5B-MLX-8bit"
)

let audio = model.synthesize(
    text: "Welcome to the demo.",
    language: "english"
)

CLI

speech speak "Welcome to the demo." \
  --engine cosyvoice \
  --cosyvoice-variant 8bit \
  --output out.wav

For zero-shot voice cloning, provide a short reference clip and transcript:

speech speak "Welcome to the demo." \
  --engine cosyvoice \
  --cosyvoice-variant 8bit \
  --voice-sample ref.wav \
  --cosy-reference-transcript "Transcript of ref.wav..." \
  --output cloned.wav

Source

Converted from FunAudioLLM/Fun-CosyVoice3-0.5B-2512.

The S3-Tokenizer-v3 PyTorch reimplementation used at conversion time is xingchensong/S3Tokenizer.

License

Apache 2.0, matching the upstream CosyVoice 3 release.

Citation

@article{du2025cosyvoice3,
  title={CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training},
  author={Du, Zhihao and others},
  journal={arXiv preprint arXiv:2505.17589},
  year={2025}
}

aufklarer/CosyVoice3-0.5B-MLX-8bit