Cue v1

💻 Code on GitHub · 📊 Full results and analysis · ✅ Check our numbers

Cue v1 is turn detection for voice agents that combines context and audio:

Cue v1 = the fine-tuned Gemma 4 E4B's logits (the words and context) × √Smart Turn v3.2 (the voice)

Combining the two gives better and faster results than either one alone. On LiveKit's eot-bench, with one model and one English prompt across 14 languages:

4th in English · 1st in Portuguese · about 13 decisions a second on one T4

Cue v1 works best with Shortwire; we'll share more details soon.

Code, every benchmark prediction and a one-command check of our numbers: github.com/PYPE-AI-MAIN/cue-v1

Thought Hold on a real call: the caller pauses mid-sentence while giving their details, the end-of-turn decision answers MID_THOUGHT (0.68), and the agent holds instead of cutting in. Recorded in Shortwire with the base Gemma 4 E4B asking this same question, before the fine-tune; Cue v1's Gemma part is that decision, trained. The phone number is masked.

What it decides

This repository holds Cue v1's Gemma part: Gemma 4 E4B, fine-tuned to answer two typed questions: when the caller pauses, are they done, and if not, why not? and when the caller speaks over the agent, should the agent stop? Each answer is read from the logits, as probabilities over named states instead of a single score, so the agent knows not only whether to act but how long to wait and whether it was really interrupted:

Context and audio together give the best results

The Gemma part brings the context: what the agent asked, what the caller has said so far, whether a sentence, a list or a phone number has been left hanging. It does not hear the voice. Smart Turn, an audio turn model, hears the voice (a falling tone, a breath) but not the meaning. Together they beat either one alone:

Across all 14 eot-bench languages, adding the voice to the Gemma part makes it faster in 12 of 14 languages at both the 5% and 10% budgets. The combination is one line, p_end × √p_voice (see Using the probabilities). Smart Turn v3.2 is free (BSD-2), from Pipecat, and is not included here: download it from pipecat-ai/smart-turn-v3.

In the results below, Cue v1 is always the combination; Gemma part only is the fine-tune on its own.

How it decides: logits, not text

Cue v1 never writes an answer. At every pause:

1. Ask one question. The decision prompt lists the options as letters, followed by what was said so far.

   • End of turn: A = DONE, B = LIKELY_DONE, C = MID_THOUGHT, D = HOLDING; with what the other side last said, what the speaker has said in this turn, and how the transcript ends.
   • Interruptions: A = STOP, B = CONTINUE; with what the agent is saying, the question it last asked, and what the caller just said over it.

2. Let the model compute one token. max_tokens: 1, with the top 20 log-probabilities returned (logprobs: true). llama.cpp returns log-probabilities, the normalised form of the logits; for choosing between options they are the same thing.

3. Read the probability of each letter and renormalise them so they sum to 1:

   Agent asked:  What's the best number to reach you on?
   Caller said:  Sure, it's nine eight four five,
   
   MID_THOUGHT 0.999   DONE 0.001   LIKELY_DONE 0.000   HOLDING 0.000
   

4. Turn the probabilities into an action, as described in Using the probabilities:

   • Caller and agent: p(DONE) + p(LIKELY_DONE) is the end-of-turn score, multiplied by the square root of an audio turn model's score.
   • Two people talking: the probabilities set how long to wait in silence.
   • Interruptions: p(STOP) decides whether the agent stops talking.

Why logits instead of an answer in words or JSON:

• Typed by construction. The answer is always one of the options. There is nothing to parse and nothing that can come back malformed.
• A full distribution, not a label. On clear cases the answer is near-certain (0.999 above). On unclear ones the probability splits between options, and that split carries into how long to wait.
• Fast. One forward pass and one token, about 100 to 300 ms on a T4.
• The fine-tune moves the logits, nothing else. Training put the loss on the answer letter only, so the model learned where to put probability (the base Gemma almost never put it on HOLDING), while staying an ordinary Gemma 4 E4B that any llama.cpp, Ollama or vLLM server can host.

Quick start (the Gemma part)

Serve the GGUF with llama.cpp:

llama-server -m cue-v1-q8_0.gguf -c 8192 -np 4 -ngl 999 --jinja --port 8090

Ask it about a pause. The answer is the first token's probability mass on each option letter; nothing is generated beyond one token.

import json, math, httpx

END_OF_TURN = json.load(open("prompts/end_of_turn_phone.json"))
INTERRUPTION = json.load(open("prompts/barge_in.json"))
LETTERS = "ABCDEFGH"
LABELS = {"agent_question": "Agent asked", "caller_earlier": "Caller earlier", "caller_text": "Caller said",
          "agent_said": "Agent is saying"}

def block(decision, inputs):
    return "\n".join(f"{LABELS.get(k, k)}: {inputs.get(k) or '(nothing)'}" for k in decision["inputs"])

def system(decision):
    ids = [o["id"] for o in decision["options"]]
    lines = [decision["instruction"], "", "Options:"]
    lines += [f"{LETTERS[i]}: {o['describe']}" for i, o in enumerate(decision["options"])]
    lines += ["", "Answer with exactly one letter."]
    if decision["examples"]:
        lines += ["", "Examples:"]
        for ex in decision["examples"]:
            lines += ["", block(decision, ex["inputs"]), f"Answer: {LETTERS[ids.index(ex['answer'])]}"]
    return "\n".join(lines)

def ask(decision, inputs):
    r = httpx.post("http://localhost:8090/v1/chat/completions", json={
        "messages": [{"role": "system", "content": system(decision)},
                     {"role": "user", "content": block(decision, inputs) + "\nAnswer:"}],
        "max_tokens": 1, "temperature": 0, "logprobs": True, "top_logprobs": 20,
        "reasoning_effort": "none", "chat_template_kwargs": {"enable_thinking": False}}).json()
    raw = {}
    for tp in r["choices"][0]["logprobs"]["content"][0]["top_logprobs"]:
        t = tp["token"].strip()
        if len(t) == 1 and t in LETTERS[:len(decision["options"])]:
            opt = decision["options"][LETTERS.index(t)]["id"]
            raw[opt] = raw.get(opt, 0.0) + math.exp(tp["logprob"])
    total = sum(raw.values()) or 1.0
    return {k: v / total for k, v in raw.items()}

probs = ask(END_OF_TURN, {"agent_question": "What's the best number to reach you on?",
                          "caller_earlier": "", "caller_text": "Sure, it's nine eight four five,",
                          "transcript_end": "no sentence end"})
print(probs)   # {'MID_THOUGHT': 0.999, 'DONE': 0.001, 'LIKELY_DONE': 0.0002, 'HOLDING': 0.0001}: half a number

Using the probabilities

Ask again whenever the transcript changes, and re-check the score as the silence grows. Until the first words of the turn are transcribed, treat the end-of-turn score as 0.5 (or the wait as 1.0 s).

Caller and agent (voice agents, phone calls)

The words decide most; an audio turn model breaks ties with what the words can't carry, such as a falling tone. We use Smart Turn v3.2 (BSD-2), rescored as the pause goes on.

p_end = probs.get("DONE", 0) + probs.get("LIKELY_DONE", 0)
score = p_end * p_voice ** 0.5          # p_voice: the audio model's end-of-turn probability for this moment
if score >= THETA:
    reply()

THETA sets the trade between replying fast and cutting callers off. Choose it on your own calls for the false cut-off rate you can accept. This is the setup behind the eot-bench numbers below: at a 5% false cut-off rate, Cue v1 replies after 719 ms on average in English.

Two people talking

Here the answer sets how long to wait. Give each option a silence budget and blend by probability:

WAITS = {"DONE": 0.5, "LIKELY_DONE": 1.0, "MID_THOUGHT": 1.75, "HOLDING": 2.25}   # seconds of silence
wait = sum(p * WAITS[o] for o, p in probs.items())
if silence_so_far >= wait:
    take_the_turn()

A confident DONE ends the turn after half a second; a likely HOLDING waits over two seconds, so someone thinking mid-story isn't cut off. Here use the Gemma part on its own, without Smart Turn: in our two-person tests, adding Smart Turn's score made it slightly worse.

Interruptions

When the caller says something while the agent is talking, ask the interruption question with what the agent is saying, the question it last asked, and the caller's words so far:

probs = ask(INTERRUPTION, {"agent_said": "...so the appointment is on Tuesday at", "agent_question": "",
                            "caller_text": "wait, Tuesday doesn't work"})
# {'STOP': 0.999, 'CONTINUE': 0.001}. The same question with "hmm" gives {'CONTINUE': 1.0, 'STOP': 0.0}
if probs.get("STOP", 0) >= 0.5:
    stop_speaking()        # a real interruption: listen
# else: a backchannel ("hmm", "okay"), keep talking

Ask again as more of the caller's words arrive: a fragment can read like a backchannel until the rest of it comes in. The threshold trades stopping on noise against talking over a caller; 0.5 is a sensible start.

Evaluation: eot-bench, English

LiveKit's eot-bench, all 400 English turns, scored with the harness's own metrics. Latency at a 5% and 10% false cut-off rate, and false cut-offs at a 300 ms and 600 ms latency budget.

Cue v1 is 4th on all four views (AUC 0.961), behind LiveKit v1, Baton v1 and Soniox, and ahead of Deepgram Flux, ultraVAD, LiveKit's local model and Smart Turn v3.2 on its own. The Gemma part on its own (AUC 0.920) is also 4th at the 5% budget and on cut-offs at 600 ms.

Honest notes on this table:

• With the base Gemma in place of the fine-tune (same prompt, same Smart Turn), the result is 723 / 502 ms: in English the fine-tune is a tie with a well-prompted base model. Its gains show on holds (below) and on two-person conversation.
• These latencies measure when Cue decides, not how fast the hardware is; see Speed and cost.

14 languages, one model, one English prompt

Cue v1 and its Gemma part on all 14 eot-bench languages, with one model and the same English prompt, English examples included: no translation, no per-language examples, no per-language tuning. The fine-tune itself only ever saw English. The one language-aware step is automatic. Where most caller turns end without punctuation in the transcript (Arabic, Hindi, Indonesian, Japanese, Korean, Chinese), the "how the transcript ends" input is set to neutral, because a missing full stop means nothing there.

Each cell is Gemma part only → Cue v1, with Cue v1's rank on the board in brackets.

Ranks are among the models the benchmark reports for that language. Not every model covers every language.

Adding the voice helps in most languages: Cue v1 is faster than its Gemma part alone in 12 of 14 languages at both budgets, and has fewer cut-offs at 600 ms in 11 of 14. In Dutch and Korean, the Gemma part alone is slightly faster at the 5% budget.

In Portuguese, Cue v1 is 1st at a 5% false cut-off rate, ahead of LiveKit v1 and Soniox (both 702 ms).

Fine-tuning on English made the other languages better too. Against Cue v1 built on the base Gemma (same prompt, same Smart Turn), the fine-tune is faster at the 5% budget in 11 of 14 languages and at the 10% budget in 11 of 14. The biggest moves: Hindi from 7th to 4th, Portuguese from 4th to 1st, French 698 to 647 ms, Korean 485 to 406 ms at 10%. It is slower in Dutch (688 to 770 ms at 5%), and at the 10% budget in English (502 to 532 ms) and Chinese (606 to 637 ms).

The whole board, with Cue v1 in the outlined column. The small rank under each Cue value counts every model eot-bench reports for that language; the outlined cell in each row is the best. Grey cells are languages a model does not support.

The other two views: latency at 10% cut-offs, and cut-offs at a 600 ms budget

Across the 14 languages, counting the 5% and 10% budgets:

Arabic is the weak spot (6th at 5%).

Speed and cost

The eot-bench latencies are not hardware speed. They measure when Cue v1 decides that the caller is done, counted from the moment the caller actually stopped. They come out the same on any hardware, as long as each decision is ready in time: the benchmark assumes words arrive 0.5 s after they're spoken, covering the ASR and the decision. On slow hardware (a CPU-only machine, a busy laptop), a late decision makes the real-world wait longer than these numbers.

Turn detection runs on Gemma logits: one token, no generation. Every decision is one forward pass that reads the option letters' probabilities, with the long prompt shared and cached across calls. Measured on one NVIDIA T4 (llama.cpp, q8_0, this repo's phone prompt, about 70 new tokens per question):

One T4 serves about 13 decisions a second; beyond two at once, questions queue. On eot-bench, Cue asked about six questions per caller turn. At an assumed six caller turns a minute, that is roughly 20 concurrent calls per T4 with every decision under 200 ms, or about $0.0005 per call-minute for turn detection on a T4 at around $0.58 an hour (an estimate: measure your own call pattern). Smart Turn v3.2, the audio part, is a small ONNX model that runs on the CPU. A faster GPU serves more.

Training data: simulated conversations

The fine-tune never saw a real recording or a benchmark transcript.

• 310 conversations written by an LLM to a written specification, each on a different topic: 150 phone calls between a caller and an agent, and 160 two-person conversations (casual, storytelling, instructional, argumentative, collaborative, transactional).
• About 10,600 labelled pauses, plus 600 backchannels and 430 real interruptions for the barge-in decision.
• Speech textures written in: fillers and false starts, repeated words, trailing conjunctions, unpunctuated endings, and numbers and addresses read out in pieces. A few Indian-English backchannels ("haan", "achha") appear, but the data is English.
• Shaped by a few aggregate statistics from TurnBench's public dev split: the share of pauses that are holds (about 35%), the share of holds that follow a complete sentence (about 70%), and the share of turn ends without punctuation (about 25%). No dev audio, transcript or label was used as a training example. The TurnBench test split and otoSpeech were not used at all.
• The loss is on the answer letter only, so the model learns where to put probability, not how to write.

On held-out simulated conversations (10% of the conversations, never trained on):

The base model almost never answers HOLDING. That is the main thing the fine-tune teaches.

Files

• Cue v1's Gemma part: Gemma 4 E4B instruction-tuned, fine-tuned with a LoRA adapter (rank 16) using Unsloth, merged and quantised to q8_0 GGUF.
• Input: text. What the other side last said, what the speaker has said so far in this turn, and how the transcript ends (a question mark, a full stop, or no sentence end). Audio is read by your ASR and, optionally, an audio turn model.
• Output: a probability for each option, from one forward pass. No JSON, no generated text, no parsing.
• Prompts are part of the model. The options, their descriptions and any worked examples live in prompts/*.json. Changing a description or adding examples in another language changes behaviour without retraining.

Intended use

Turn-taking inside voice agents and other spoken-dialogue systems that already have an ASR: deciding when to reply, when to keep listening, and whether something the caller says over the agent should stop it.

Limits

• The Gemma part reads words, not prosody. On its own it cannot hear a falling tone or a breath; that is what Smart Turn adds in Cue v1. Using the Gemma part alone gives the slower "Gemma part only" numbers above.
• Short objections can pass as backchannels. "Hmm, actually no" said over the agent comes back CONTINUE (0.94). Clear interruptions ("wait, Tuesday doesn't work") are STOP at 0.999, but a lone "no" or "actually" can be missed.
• Needs an ASR and a GPU. It is not an on-device model.
• Trained on English only. It carries over to 13 other languages with the English prompt, but every language except Portuguese still has a faster hosted model, and Arabic is behind most of the board.
• Trained on simulated conversations. Real speech is messier than any written transcript. Measure it on your own calls before relying on its probabilities.

Part of Shortwire

Cue is the turn engine in Shortwire, Pype's open source voice AI stack. Shortwire splits a call in two:

• Background decision models handle how the call moves: when the caller is done, when to hold, when to stop talking, what to guard against, when a number is being dictated, which tool to call.
• The foreground LLM focuses on the talking: what the agent says.

Every decision the cascade makes is auditable: each one is logged with its answer and probability, and can be replayed after the call. In Shortwire, MID_THOUGHT and HOLDING are what we call Thought Hold, and the record of every decision is the Decision Trace.

Knows when to wait. Knows when to stop.

Cue v1 works best with Shortwire. We'll share more details soon.

Licence

Apache 2.0, the same as the base model, Gemma 4 E4B.

Citation

@misc{pype2026cue,
  title  = {Cue v1: turn detection from context and audio, with a fine-tuned decision model},
  author = {Pype},
  year   = {2026},
  url    = {https://huggingface.co/Pype/cue-v1}
}