JamePeng2023/Qwen3-TTS-12Hz-Base-GGUF

The Qwen3-TTS is now supported in llama-cpp-python. This project provides a test GGUF file.

llama-cpp-python: https://github.com/JamePeng/llama-cpp-python

Code example:

from contextlib import closing

from llama_cpp import Llama, LLAMA_POOLING_TYPE_NONE
from llama_cpp.llama_multimodal import MTMDAudioGenerator

with closing(Llama(
    model_path="/path/to/model/qwen3-TTS-12Hz-1.7B-Base-BF16.gguf",
    embeddings=True,
    pooling_type=LLAMA_POOLING_TYPE_NONE,
    n_ctx=4096,
    n_gpu_layers=-1,  # Use 0 for CPU inference.
)) as llama:
    with MTMDAudioGenerator(
        mmproj_path="/path/to/model/mmproj-qwen3-TTS-12Hz-1.7B-Base-BF16.gguf",
        use_gpu=True,
        flash_attn=None,  # AUTO; True enables FA, False disables it.
    ) as generator:
        audio = generator.create_speech(
            llama=llama,
            text="Hello, welcome to speech synthesis.",
            language="en",
            speaker_reference="/path/to/reference.wav",
            seed=42,
        )
        audio.save("output.wav")