ONNX export of Qwen/Qwen3-TTS-12Hz-0.6B-Base for voice cloning with ONNX Runtime. No PyTorch required at inference time.
Provide a short reference audio clip and its transcript, and the model synthesizes new text in the same voice using In-Context Learning (ICL).
Both FP32 and INT4 (weight-only, RTN) variants are included.
Exported and maintained by WaveKat as part of the wavekat-tts voice pipeline.
pip install -r requirements.txt
# FP32
python generate_clone_onnx.py \
--ref-audio ref.wav --ref-text "Transcript of the reference audio." \
--text "New text to synthesize in the cloned voice." \
-o output_fp32.wav
# INT4 (~4x smaller, faster)
python generate_clone_onnx.py --variant int4 \
--ref-audio ref.wav --ref-text "Transcript of the reference audio." \
--text "New text to synthesize in the cloned voice." \
-o output_int4.wav
Reference audio should be mono 24 kHz WAV with a clear, single-speaker recording (3–10 seconds works well). The script will resample automatically via librosa if needed.
Qwen3-TTS 0.6B Base uses a 6-model voice-clone pipeline:
Ref audio --> [Speaker Encoder] ECAPA-TDNN → 1024-d speaker embedding
\-> [Tokenizer Encoder] Mimi encoder → 16-group ref codes (12 Hz)
Text + Ref text + Speaker embed + Ref codes
|
v (ICL prefill)
[Talker LM] 28 layers, 1024 hidden
predicts codebook group 0
|
v
[Code Predictor] 5 layers, 1024 hidden
predicts groups 1-15
|
v
[Vocoder] single forward pass
concat(ref_codes, gen_codes) → 24 kHz waveform → trim ref portion
The pipeline is split into 6 ONNX models:
| Model | Description | Precision |
|---|---|---|
speaker_encoder.onnx |
ECAPA-TDNN: mel → 1024-d speaker embedding | FP32 only |
tokenizer_encoder.onnx |
Mimi encoder: audio → 16-group codec codes | FP32 only |
talker_prefill.onnx |
Full sequence prefill with KV cache output | FP32 / INT4 |
talker_decode.onnx |
Single-step decode with KV cache | FP32 / INT4 |
code_predictor.onnx |
Predict codebook groups 1-15 | FP32 / INT4 |
vocoder.onnx |
Codes to 24 kHz waveform | FP32 / INT4 |
Speaker encoder and tokenizer encoder are always FP32 — they run once per request and are small.
.
├── config.json # Model config (dimensions, token IDs, language map)
├── speaker_encoder.onnx # ECAPA-TDNN speaker encoder (FP32)
├── tokenizer_encoder.onnx # Mimi speech tokenizer encoder (FP32)
├── tokenizer/ # Text tokenizer (vocab, merges)
├── embeddings/ # Pre-extracted embedding weights (.npy)
├── fp32/ # FP32 ONNX models
│ ├── talker_prefill.onnx
│ ├── talker_decode.onnx
│ ├── code_predictor.onnx
│ └── vocoder.onnx
├── int4/ # INT4 weight-only quantized models
│ ├── talker_prefill.onnx
│ ├── talker_decode.onnx
│ ├── code_predictor.onnx
│ └── vocoder.onnx
├── generate_clone_onnx.py # Reference ONNX-only voice clone script
└── requirements.txt # Inference dependencies
English, Chinese, Japanese, Korean, German, French, Spanish, Italian, Portuguese, Russian.
The export scripts are in the wavekat-tts repository:
cd tools/qwen3-tts-onnx
pip install -r requirements.txt
# Export FP32, quantize INT4, and package for HF
make clone-all
WaveKat builds open-source voice pipeline components in Rust. This ONNX export is maintained as part of wavekat-tts, which provides unified TTS inference across multiple backends.
int4/talker_prefill.onnx 4 MBint4/talker_decode.onnx 4 MBfp32/vocoder.onnx 3 MBint4/vocoder.onnx 3 MBfp32/talker_prefill.onnx 3 MBfp32/talker_decode.onnx 3 MBtokenizer_encoder.onnx 553.9 kBint4/code_predictor.onnx 360.5 kBfp32/code_predictor.onnx 250.1 kBspeaker_encoder.onnx 218.3 kB