Ming-3.0-tts-0.5B — int8 weight-only
An int8 weight-only quantization of inclusionAI/Ming-3.0-tts-0.5B. The checkpoint is 1.98 GB instead of 2.96 GB (67%), with no audible quality loss in side-by-side listening tests.
| BF16 (original) | int8 (this repo) | |
|---|---|---|
| Weights on disk | 2.96 GB | 1.98 GB |
| Runtime memory (Apple MPS, one request) | 4.27 GB | 3.25 GB |
What is quantized
Every nn.Linear weight with at least 65,536 parameters in these modules is stored as int8 with one float32 scale per output channel (symmetric, round-to-nearest, no calibration data):
model.model.layers.*: Qwen2.5-0.5B backboneaudio.encoder.*: audio VAE encoder (encodes the voice-cloning prompt)flowloss.cfm.*: flow-matching DiT headlinear_proj_audio.*: audio aggregator
These stay in BF16:
audio.decoder.*: audio VAE decoder / vocoder, the part most sensitive to artifactsmodel.model.embed_tokens: tied tolm_head- Norms, biases and small projections
Mean weight SNR is 41.3 dB (worst tensor 32.0 dB). At runtime the weights stay int8 in memory and are dequantized layer by layer during the forward pass.
Evaluation
These are informal listening tests on CPU with the same seed, text and settings for both checkpoints:
| Test | Result |
|---|---|
FREEFORM English storytelling (cookbook example) |
int8 and BF16 indistinguishable by ear |
TTS zero-shot voice clone, English, 6 s reference |
int8 closely matches the reference voice, same as BF16 |
No formal benchmarks (WER, SIM, MOS) have been run on this checkpoint.
Usage
Inference needs the official Ming-3.0-tts code plus int8_runtime.py from this repo. MingTTSInt8 is a drop-in replacement for MingTTS.
git clone https://github.com/inclusionAI/Ming-3.0-tts.git
cd Ming-3.0-tts
pip install -r requirements.txt
hf download ahmadw/Ming-3.0-tts-0.5B-int8 int8_runtime.py --local-dir .
from int8_runtime import MingTTSInt8
tts = MingTTSInt8("ahmadw/Ming-3.0-tts-0.5B-int8", device="cuda:0") # or "cpu"
tts.synthesize(
task="TTS",
language_id="English",
text="Every morning brings a new chance to begin again.",
prompt_wav_path="reference.wav",
prompt_text="Exact transcript of reference.wav.",
use_spk_emb=True,
output_path="out.wav",
)
All tasks of the base model (TTS, FREEFORM, VOICE_DESIGN, MULTILINGUAL, BASIC, EMOTION, DIALECT, IP) work unchanged; see the task cookbook.
Tips
- Voice-cloning prompts: use 4 to 8 seconds of one calm, clean sentence, with a transcript that matches the audio word for word. Long, fast or shouted promo clips make the clone rushed and change the voice; this happens with the BF16 model too.
- Non-CUDA devices: the upstream code enables BF16 autocast only on CUDA.
MingTTSInt8adds a cast onspk_headso voice cloning also runs on CPU. Apple MPS is not verified; CPU is. - Speed: int8 saves size and memory, but it is not faster. On CPU it is slower than BF16 (about 30 s against 17 s for a 5 s clip), because weights are dequantized on every call.
Reproducing
python quantize_int8.py --src /path/to/Ming-3.0-tts-0.5B --dst ./int8
# --include-decoder also quantizes the vocoder (about 1.27 GB, higher artifact risk)
License and credits
MIT, the same as the base model. All credit for the model goes to the Ming-3.0-tts authors at inclusionAI / Ant Group; this repo only adds weight quantization.