Lure 0.6B
Expressive voice cloning in 0.6B parameters. It laughs, sighs, breaks and whispers where you tell it to.
Research preview · English · 0.6B parameters · 24 kHz
[!WARNING] Research preview. Weights, behaviour and this card can change between releases. Pin a revision if you build on it.
Overview
Lure is a 0.6B text-to-speech model for English. Give it a few seconds of a voice and the transcript of that clip, and it
reads new text in that voice. A sound written inline, like (sigh) or (sharp inhale), is performed at that spot
instead of read out. A short acting note, like "Hushed and shaking, close to tears.", changes how the line is played.
Most open models in its class are 1.5B to 35B parameters. On the public English benchmarks below, Lure's word accuracy sits alongside models three to sixty times its size, and its naturalness score (UTMOS) is the highest in the table. It runs faster than real time on a MacBook, and a single 24 GB GPU streams 8 voices at once.
Lure Turbo is the same voice with a faster decoding head. It predicts the 15 fine audio codes of each frame in 3 passes instead of 15, so the per-frame decode step is about 3.4x faster, and its benchmark scores match Lure's. See Speed.
| Parameters | 0.6B |
| Language | English only |
| Output | 24 kHz mono, streaming |
| Codec | 12.5 Hz, 16 codebooks (Qwen3-TTS 12Hz format) |
| Cloning | zero-shot, 3 to 12 s reference plus its transcript |
| Sounds | inline tags in round brackets, 36 supported |
| Direction | optional free-text acting note |
Speed
Lure streams audio while it generates. RTF is generation time divided by audio length, so lower is faster. All numbers come from the lure-tts engine with the cuda backend in bf16.
H100
| Model | Streams | ms per frame | RTF |
|---|---|---|---|
| Lure | 1 | 18.4 | 0.23 |
| Lure | 8 | 19.6 | 0.25 |
| Lure Turbo | 1 | 11.4 | 0.14 |
| Lure Turbo | 8 | 12.4 | 0.15 |
24 GB GPU
| Model | 1 stream | 8 streams, total |
|---|---|---|
| Lure | 28.8 frames/s (2.3x real time) | 219 frames/s (17x real time) |
| Lure Turbo | 50.6 frames/s (4.0x real time) | 374 frames/s (30x real time) |
Benchmarks
We ran Lure and Lure Turbo with each benchmark's official scripts, one take per line and default sampling. The other rows are the numbers their developers published. The best value in each column is in bold.
| Model | Size | Seed-TTS WER ↓ | Seed-TTS SIM ↑ | LibriSpeech-PC SIM ↑ | LibriSpeech-PC UTMOS ↑ | VDB SIM ↑ | CV3 emotion % ↑ | RTF ↓ |
|---|---|---|---|---|---|---|---|---|
| Lure | 0.6B | 1.51 | 0.653 | 0.638 | 4.43 | 0.81 | 73.3 | 0.230 |
| Lure Turbo | 0.6B | 1.59 | 0.654 | 0.641 | 4.38 | 0.81 | 73.3 | 0.140 |
| Fish S2 Pro | 5B | 0.99 | – | – | – | – | – | 0.195 |
| Qwen3-TTS 0.6B | 0.6B | 1.32 | – | – | – | – | – | – |
| Fish Drama 3 | 35B MoE | 1.54 | 0.640 | 0.605 | 4.31 | – | – | 0.131 |
| Ming 3.0 0.5B | ~1B | 1.58 | 0.691 | – | – | – | – | – |
| FireRedTTS3 | 1.7B+ | 1.64 | 0.772 | – | – | – | – | – |
| VoxCPM2 | 2B | 1.84 | 0.753 | – | – | 0.70 | – | – |
| F5-TTS | – | 2.00 | 0.670 | – | – | – | 64.7 | – |
| CosyVoice3 | – | 2.22 | 0.720 | 0.694 | 4.28 | – | – | – |
| Breeze TTS 2 | 3B | – | – | – | – | 0.67 | – | 0.320 |
| dots.tts | 2B | – | 0.770 | – | – | – | – | – |
| Ming Tiny | 7.9B MoE | – | 0.728 | – | – | – | 74.0 | – |
| IndexTTS2 | 1.5B | – | 0.706 | – | – | – | – | – |
| OmniVoice | 0.6B | – | – | 0.729 | 4.28 | – | – | – |
| Qwen3-TTS | 0.6B-1.7B | – | – | 0.704 | 4.41 | – | – | – |
| Real recordings | – | – | – | – | 4.10 | – | – | – |
| Inworld TTS-2 | – | – | – | – | – | 0.71 | – | – |
| MiMo-v2.5 | – | – | – | – | – | 0.66 | – | – |
| Qwen-Audio 3.0+ | – | – | – | – | – | 0.65 | – | – |
| StepAudio 2.5 | – | – | – | – | – | 0.64 | – | – |
| CosyVoice3 1.5B +DiffRO-EMO | 1.5B | – | – | – | – | – | 77.7 | – |
| Ming 0.5B | ~1B | – | – | – | – | – | 72.3 | – |
| CosyVoice3 0.5B | 0.5B | – | – | – | – | – | 66.3 | – |
| CosyVoice3 1.5B | 1.5B | – | – | – | – | – | 63.0 | – |
| Higgs TTS 3 | 4B | – | – | – | – | – | – | 0.147 |
Quick start
lure-tts (recommended: streaming, Turbo, Mac / NVIDIA / CPU)
git clone https://github.com/smirki/lure-tts && cd lure-tts
hf download Tesslate/Lure-TTS-0.6B --local-dir models/lure
pip install ./backends/mlx ".[server]" # Mac; see the repo for NVIDIA and the C++ build
from lure_tts import Lure
tts = Lure.load("models/lure")
voice = tts.make_voice("reference.wav", "Transcript of the reference clip.", name="me")
tts.speak("Wait, you're staying? (sharp inhale) Since when?", voice,
instruct="Sudden, bright shock, then a caught breath.", out="out.wav")
for pcm in tts.stream("(sigh) Fine. I'm in.", "me"): # float32 chunks at 24 kHz
...
lure serve starts an OpenAI-compatible speech server (POST /v1/audio/speech) that batches concurrent requests.
qwen-tts (PyTorch)
Lure uses the Qwen3-TTS 12Hz format, so the qwen-tts package loads it directly (without Turbo):
import soundfile as sf
from qwen_tts import Qwen3TTSModel
tts = Qwen3TTSModel.from_pretrained("Tesslate/Lure-TTS-0.6B", device_map="cuda:0", dtype="bfloat16")
wavs, sr = tts.generate_voice_clone(
text="I'm sorry. (sigh) I never meant for any of this to happen.",
language="English",
ref_audio="reference.wav",
ref_text="Transcript of the reference clip.",
non_streaming_mode=True,
)
sf.write("out.wav", wavs[0], sr)
qwen-tts turns the direction (instruct) off for 0.6B models; use lure-tts if you want directions.
Sound tags
Write tags in round brackets, lowercase, at the spot where the sound should happen:
I can't believe it (laugh) you actually did it. Square brackets like [laugh] are not supported.
| Group | Tags |
|---|---|
| Laughter | (laugh) (chuckle) (giggle) (nervous laugh) (wheezing laugh) (laughs while talking) |
| Breath | (sigh) (frustrated sigh) (relieved sigh) (exhale) (inhale) (sharp inhale) (gasp) (happy gasp) (yawn) |
| Crying | (voice breaking) (sob) (whimper) (sniff) (sniffle) |
| Throat and mouth | (cough) (clears throat) (swallow) (gulp) (lip smack) (tsk) (snort) (grunt) (groan) (hum) |
| Spoken sounds | (hmm) (ugh) (pfft) (scoff) (ow) |
| Timing | (pause) |
- Laughs, sighs, gasps and sharp inhales are the most reliable. Rarer tags like
(ow)or(sob)land less often. - One or two tags per sentence works best.
- If a take misses a tag, regenerate with a new seed.
Direction
A direction is a short note to the actor about emotion, pace, pitch or breath. It is optional. Without one, Lure takes the delivery from the words and tags.
Sudden, bright shock. Fast attack on the question, then a caught breath lifting pitch sharply upward.Slow, stunned, hushed. Barely any pitch movement, like the brain is still buffering.Tight, shivering disbelief. Thin voice, trembly pitch, the laugh escaping between clauses.
Recommended settings
| Setting | Value |
|---|---|
| Language | english (always set it; auto uses a different prompt and hurts quality) |
| Sampling | temperature 0.9, top-k 50, top-p 1.0, repetition penalty 1.05, same for the sub-talker |
| Precision | bf16 |
| Prompt layout | non-streaming (full text up front); lure-tts does this automatically |
| Reference clip | 3 to 12 s, one speaker, clean, no music. The output inherits the clip's mic and room. |
| Reference transcript | The exact words in the clip. A wrong transcript causes skipped or repeated words. |
| Text per call | A sentence or short paragraph. For long text, split on sentences (lure-tts does this and streams each piece). |
| Retry rule | If a take runs past twice its expected length or never stops, regenerate with a new seed. |
Limitations
- English only. Other languages have not been tested.
- Directions shape emotion and pace well, but Lure is weaker than large instruction-tuned models at changing who is speaking (accent, age, gender) on request.
- Sound tags are best effort; a take sometimes skips one.
- Very long single calls can stop early or drift in voice. Split long text into sentences.
- Do not use Lure to imitate a real person without their consent.
Training data includes YODAS v3.
License
Apache 2.0 for everything: the weights, the lure-tts engine and the lure-train training code.