Qwen3-TTS GGUF + ONNX
Pre-converted weights for running Qwen3-TTS 12Hz 1.7B Base entirely via llama.cpp + ONNX Runtime. No PyTorch required.
Which model is this: the 1.7B Base variant (voice cloning from a reference sample), not the 0.6B, and not CustomVoice or VoiceDesign. About 1.5 GB total.
Why this repo
The upstream conversion toolchain expects the original PyTorch weights and a three-stage export (codec/speaker/decoder ONNX, talker GGUF, predictor GGUF) spread across a dozen scripts. These files are the finished output of that pipeline: download and run.
Files
| File | Size | Format |
|---|---|---|
| qwen3_tts_talker.q5_k.gguf | 960 MB | GGUF Q5_K |
| qwen3_tts_predictor.q8_0.gguf | 145 MB | GGUF Q8_0 |
| qwen3_tts_decoder.fp16.onnx | 220 MB | ONNX FP16 |
| qwen3_tts_speaker_encoder.fp16.onnx | 24 MB | ONNX FP16 |
| qwen3_tts_codec_encoder.fp16.onnx | 109 MB | ONNX FP16 |
| tokenizer.json | 11 MB | JSON |
INT8 and FP32 variants are also included for the ONNX components.
Usage
Requires the HaujetZhao/Qwen3-TTS-GGUF runtime package and llama.cpp compiled libraries.
pip install huggingface_hub
hf download Solidaxel/Qwen3-TTS-GGUF --local-dir model-base
Then point the runtime at that directory:
from qwen3_tts_gguf import TTSEngine, TTSConfig
engine = TTSEngine(model_dir="model-base")
stream = engine.create_stream()
stream.set_voice("output/elaborate/sample.json") # reference voice
result = stream.clone("Hello, world!", config=TTSConfig(temperature=0.8, seed=42))
stream.join()
result.save("output.wav")
The runtime also ships 51-Interactive-Clone.py for an interactive session and 43-Inference-Base.py for the Base variant, which is the one in this repo.
Credits
- Original model: Qwen/Qwen3-TTS-12Hz-1.7B-Base (Apache 2.0)
- GGUF/ONNX conversion: HaujetZhao/Qwen3-TTS-GGUF (MIT)
- Redistribution: SolidAxel Project
License
Apache 2.0, same as the original model.