Lure 0.6B

Expressive voice cloning in 0.6B parameters. It laughs, sighs, breaks and whispers where you tell it to.
_{Research preview · English · 0.6B parameters · 24 kHz}

[!WARNING] Research preview. Weights, behaviour and this card can change between releases. Pin a revision if you build on it.

Overview

Lure is a 0.6B text-to-speech model for English. Give it a few seconds of a voice and the transcript of that clip, and it reads new text in that voice. A sound written inline, like (sigh) or (sharp inhale), is performed at that spot instead of read out. A short acting note, like "Hushed and shaking, close to tears.", changes how the line is played.

Most open models in its class are 1.5B to 35B parameters. On the public English benchmarks below, Lure's word accuracy sits alongside models three to sixty times its size, and its naturalness score (UTMOS) is the highest in the table. It runs faster than real time on a MacBook, and a single 24 GB GPU streams 8 voices at once.

Lure Turbo is the same voice with a faster decoding head. It predicts the 15 fine audio codes of each frame in 3 passes instead of 15, so the per-frame decode step is about 3.4x faster, and its benchmark scores match Lure's. See Speed.


Parameters	0.6B
Language	English only
Output	24 kHz mono, streaming
Codec	12.5 Hz, 16 codebooks (Qwen3-TTS 12Hz format)
Cloning	zero-shot, 3 to 12 s reference plus its transcript
Sounds	inline tags in round brackets, 36 supported
Direction	optional free-text acting note

Speed

Lure streams audio while it generates. RTF is generation time divided by audio length, so lower is faster. All numbers come from the lure-tts engine with the cuda backend in bf16.

H100

Model	Streams	ms per frame	RTF
Lure	1	18.4	0.23
Lure	8	19.6	0.25
Lure Turbo	1	11.4	0.14
Lure Turbo	8	12.4	0.15

24 GB GPU

Model	1 stream	8 streams, total
Lure	28.8 frames/s (2.3x real time)	219 frames/s (17x real time)
Lure Turbo	50.6 frames/s (4.0x real time)	374 frames/s (30x real time)

Benchmarks

We ran Lure and Lure Turbo with each benchmark's official scripts, one take per line and default sampling. The other rows are the numbers their developers published. The best value in each column is in bold.

Model	Size	Seed-TTS WER ↓	Seed-TTS SIM ↑	LibriSpeech-PC SIM ↑	LibriSpeech-PC UTMOS ↑	VDB SIM ↑	CV3 emotion % ↑	RTF ↓
Lure	0.6B	1.51	0.653	0.638	4.43	0.81	73.3	0.230
Lure Turbo	0.6B	1.59	0.654	0.641	4.38	0.81	73.3	0.140
Fish S2 Pro	5B	0.99	–	–	–	–	–	0.195
Qwen3-TTS 0.6B	0.6B	1.32	–	–	–	–	–	–
Fish Drama 3	35B MoE	1.54	0.640	0.605	4.31	–	–	0.131
Ming 3.0 0.5B	~1B	1.58	0.691	–	–	–	–	–
FireRedTTS3	1.7B+	1.64	0.772	–	–	–	–	–
VoxCPM2	2B	1.84	0.753	–	–	0.70	–	–
F5-TTS	–	2.00	0.670	–	–	–	64.7	–
CosyVoice3	–	2.22	0.720	0.694	4.28	–	–	–
Breeze TTS 2	3B	–	–	–	–	0.67	–	0.320
dots.tts	2B	–	0.770	–	–	–	–	–
Ming Tiny	7.9B MoE	–	0.728	–	–	–	74.0	–
IndexTTS2	1.5B	–	0.706	–	–	–	–	–
OmniVoice	0.6B	–	–	0.729	4.28	–	–	–
Qwen3-TTS	0.6B-1.7B	–	–	0.704	4.41	–	–	–
Real recordings	–	–	–	–	4.10	–	–	–
Inworld TTS-2	–	–	–	–	–	0.71	–	–
MiMo-v2.5	–	–	–	–	–	0.66	–	–
Qwen-Audio 3.0+	–	–	–	–	–	0.65	–	–
StepAudio 2.5	–	–	–	–	–	0.64	–	–
CosyVoice3 1.5B +DiffRO-EMO	1.5B	–	–	–	–	–	77.7	–
Ming 0.5B	~1B	–	–	–	–	–	72.3	–
CosyVoice3 0.5B	0.5B	–	–	–	–	–	66.3	–
CosyVoice3 1.5B	1.5B	–	–	–	–	–	63.0	–
Higgs TTS 3	4B	–	–	–	–	–	–	0.147

Quick start

lure-tts (recommended: streaming, Turbo, Mac / NVIDIA / CPU)

git clone https://github.com/smirki/lure-tts && cd lure-tts
hf download Tesslate/Lure-TTS-0.6B --local-dir models/lure
pip install ./backends/mlx ".[server]"        # Mac; see the repo for NVIDIA and the C++ build

from lure_tts import Lure

tts = Lure.load("models/lure")
voice = tts.make_voice("reference.wav", "Transcript of the reference clip.", name="me")
tts.speak("Wait, you're staying? (sharp inhale) Since when?", voice,
          instruct="Sudden, bright shock, then a caught breath.", out="out.wav")

for pcm in tts.stream("(sigh) Fine. I'm in.", "me"):    # float32 chunks at 24 kHz
    ...

lure serve starts an OpenAI-compatible speech server (POST /v1/audio/speech) that batches concurrent requests.

qwen-tts (PyTorch)

Lure uses the Qwen3-TTS 12Hz format, so the qwen-tts package loads it directly (without Turbo):

import soundfile as sf
from qwen_tts import Qwen3TTSModel

tts = Qwen3TTSModel.from_pretrained("Tesslate/Lure-TTS-0.6B", device_map="cuda:0", dtype="bfloat16")
wavs, sr = tts.generate_voice_clone(
    text="I'm sorry. (sigh) I never meant for any of this to happen.",
    language="English",
    ref_audio="reference.wav",
    ref_text="Transcript of the reference clip.",
    non_streaming_mode=True,
)
sf.write("out.wav", wavs[0], sr)

qwen-tts turns the direction (instruct) off for 0.6B models; use lure-tts if you want directions.

Sound tags

Write tags in round brackets, lowercase, at the spot where the sound should happen: I can't believe it (laugh) you actually did it. Square brackets like [laugh] are not supported.

Group	Tags
Laughter	`(laugh)` `(chuckle)` `(giggle)` `(nervous laugh)` `(wheezing laugh)` `(laughs while talking)`
Breath	`(sigh)` `(frustrated sigh)` `(relieved sigh)` `(exhale)` `(inhale)` `(sharp inhale)` `(gasp)` `(happy gasp)` `(yawn)`
Crying	`(voice breaking)` `(sob)` `(whimper)` `(sniff)` `(sniffle)`
Throat and mouth	`(cough)` `(clears throat)` `(swallow)` `(gulp)` `(lip smack)` `(tsk)` `(snort)` `(grunt)` `(groan)` `(hum)`
Spoken sounds	`(hmm)` `(ugh)` `(pfft)` `(scoff)` `(ow)`
Timing	`(pause)`

Laughs, sighs, gasps and sharp inhales are the most reliable. Rarer tags like (ow) or (sob) land less often.
One or two tags per sentence works best.
If a take misses a tag, regenerate with a new seed.

Direction

A direction is a short note to the actor about emotion, pace, pitch or breath. It is optional. Without one, Lure takes the delivery from the words and tags.

Sudden, bright shock. Fast attack on the question, then a caught breath lifting pitch sharply upward.
Slow, stunned, hushed. Barely any pitch movement, like the brain is still buffering.
Tight, shivering disbelief. Thin voice, trembly pitch, the laugh escaping between clauses.

Recommended settings

Setting	Value
Language	`english` (always set it; `auto` uses a different prompt and hurts quality)
Sampling	temperature 0.9, top-k 50, top-p 1.0, repetition penalty 1.05, same for the sub-talker
Precision	bf16
Prompt layout	non-streaming (full text up front); lure-tts does this automatically
Reference clip	3 to 12 s, one speaker, clean, no music. The output inherits the clip's mic and room.
Reference transcript	The exact words in the clip. A wrong transcript causes skipped or repeated words.
Text per call	A sentence or short paragraph. For long text, split on sentences (lure-tts does this and streams each piece).
Retry rule	If a take runs past twice its expected length or never stops, regenerate with a new seed.

Limitations

English only. Other languages have not been tested.
Directions shape emotion and pace well, but Lure is weaker than large instruction-tuned models at changing who is speaking (accent, age, gender) on request.
Sound tags are best effort; a take sometimes skips one.
Very long single calls can stop early or drift in voice. Split long text into sentences.
Do not use Lure to imitate a real person without their consent.

Training data includes YODAS v3.

License

Apache 2.0 for everything: the weights, the lure-tts engine and the lure-train training code.