Awaken: Efficient Full-Duplex Adaptation with Capability Retention
Full-duplex voice models can listen and speak at the same time, so they can acknowledge, interrupt, and hand over turns the way people do. Teaching this usually depends on natural conversational audio rich in backchannels. That audio is scarce, and without careful curation it can weaken a model's reasoning and grounding. Awaken is based on FunAudioChat, adapting the half-duplex model for full-duplex conversation using only a small amount of dialogue data while retaining most of the base model's benchmark performance.
See the project page for the full data composition and training details.
| Attribute | Value |
|---|---|
| Model Size | ~8B parameters |
| Base Model | Fun-Audio-Chat-8B (half-duplex) |
| Backbone | Qwen3-8B, one position per 200 ms frame |
| Input | Streaming 24 kHz user audio through a causal Whisper-style encoder (2 s attention window) |
| Output | Text tokens and 5 codec tokens per frame, rendered by the CosyVoice3 flow decoder and HiFT vocoder |
| Interaction | Full-duplex: no turns, no voice-activity detector |
| Languages | English |
| License | Apache 2.0 (same as the base model) |
git clone https://github.com/avera-labs/AwakenFullDuplex-FunAudioChat
cd AwakenFullDuplex-FunAudioChat
# 1. torch + torchaudio for YOUR CUDA driver first (the default PyPI wheel targets the newest
# CUDA and will not load on older drivers), e.g. for CUDA 12.8:
pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu128
# 2. the package
pip install -e . # Linux + CUDA GPU; pulls transformers, tensorrt, ...
pip install -e ".[fast]" # + flash-attn (faster decode; optional)
pip install -e ".[voice]" # + onnxruntime, to make voice prompts from a clip
pip install huggingface-hub
hf download openmartdev/AwakenFullDuplex-FunAudioChat --local-dir ./pretrained_models/awaken-chat
The speech codec is fetched automatically from
FunAudioLLM/Fun-CosyVoice3-0.5B-2512
(flow, vocoder and the estimator's ONNX export only). The TensorRT engine is built once per
machine on first use, which takes a few minutes.
Live server (one conversation at a time, WebSocket protocol in awaken_chat/server/protocol.py):
awaken-server --model ./pretrained_models/awaken-chat --port 8998 [--ssl-dir certs] [--voices-dir assets/voices]
Offline, with a recorded user track:
awaken-offline --model ./pretrained_models/awaken-chat --input user.wav --out out/ # stereo.wav, transcript.txt, tokens.pt
Make a voice prompt from a reference clip (3-10 s of one speaker you have the rights to):
awaken-voice --clip speaker.wav --out assets/voices/speaker.pt
From Python:
from awaken_chat import load_model, Decoders, DuplexSession, system_prompt
loaded = load_model("./pretrained_models/awaken-chat") # HF repo id or local directory
decoders = Decoders(loaded.model) # static caches + CUDA graphs, once per process
session = DuplexSession(loaded.model, decoders, loaded.prompt_ids(system_prompt("You are a tour guide.")))
for features in ear_frames: # (5, 4096) per 200 ms, see models/ear.py
out = session.step(features) # out.text_id, out.audio_ids, out.state
We measure listener behavior, following the conversation, and knowledge and reasoning side by side. Awaken and Fun-Audio-Chat-8B are run by us; other rows come from the public InteractionBench results and the Fun-Audio-Chat technical report.
| System | Backchannel pass ↑ | Floor-take ↓ | Groundedness probes ↑ | Logic puzzles ↑ | Llama Q. ↑ | TriviaQA ↑ |
|---|---|---|---|---|---|---|
| GPT-Realtime | 10% | 5% | 67% | 100% | – | – |
| Moshi | 78% | 22% | 14% | 30% | 65.67 | 18.46 |
| Freeze-Omni | 0% | 39% | 35% | 40% | 74.00 | 30.08 |
| PersonaPlex | 83% | 11% | 30% | 30% | – | – |
| Fun-Audio-Chat-Duplex-8B | – | – | – | – | 72.33 | 29.59 |
| Fun-Audio-Chat-8B (half-duplex base) | n/a | n/a | 66% | 100% | 76.2 | 48.7 |
| Awaken SFT | 1% | 6% | 62% | 87% | 70.7 | 35.5 |
| Awaken GRPO | 63% | 3% | 61% | 90% | 73.4 | 36.2 |
| System | OpenBookQA ↑ |
|---|---|
| Fun-Audio-Chat-8B (half-duplex) | 83.52 |
| Fun-Audio-Chat-Duplex-8B (reported) | – |
| Freeze-Omni (reported) | 30.98 |
| Moshi (reported) | 25.93 |
| Awaken SFT | 76.2 |
| Awaken GRPO | 75.6 |
Because our runs and the published results are scored differently, compare across systems with care. Full methodology is on the project page.
Every 200 ms ("one frame") the model consumes
and predicts the next text token and the next five codec tokens. The text lane carries the
words and two control tokens: pad means "listening" and, during speech, "I am done" (the
release); sil means "words done, audio still finishing". The codec tokens are rendered to
speech by CosyVoice3's flow-matching decoder and HiFT vocoder.
mic 24 kHz ─► StreamingEar ─► 5 feature vectors per frame ─┐
▼
own text token ──► Qwen3-8B backbone (one position per frame) ──► next text token
own codec tokens ─┘ │
▼
depth decoder (5 codec tokens / frame) ──► CosyVoice3 flow + HiFT ──► speaker
Both transformer stacks run through a fused, CUDA-graphed single-token decoder with static
key/value caches (awaken_chat/inference/static_decoder.py), so a frame costs a few
milliseconds of launch overhead instead of thousands of small kernels.
DecodeParams (awaken_chat/inference/params.py) has two presets. bench is the recipe the
reported numbers were produced with (text T 0.7 / top-k 25, audio T 0.8 / top-k 250).
live is sharper inside a text burst (T 0.2 / top-k 5), penalizes repeated content tokens and
caps consecutive near-silent codec tokens; it is what the server uses by default. Any field
can be overridden per conversation in the start message.
awaken_chat/
frame.py frame rate, group size, sample rates, special token ids, system prompt
models/ configuration, ear, token embedder, depth decoder, AwakenModel, loaders
inference/ static decoder, session state machine, sampling, decode params, offline runner
codec/ CosyVoice3 flow + HiFT (vendored, trimmed), streaming detokenizer, worker process, voices
server/ WebSocket protocol, live session runner, FastAPI app
cli.py awaken-server / awaken-offline / awaken-voice
scripts/ ws_client.py, a paced WebSocket client that drives the server from a wav file
tests/ CPU unit tests on tiny random models
hf_model_card/ Hugging Face model card (README + figures)
index.html project page
The code and model weights are licensed under the Apache 2.0 License, the same license as the base model Fun-Audio-Chat-8B.
The model architecture is derived from Fun-Audio-Chat
and the speech codec from CosyVoice (both Apache-2.0);
the text backbone is Qwen3, and the ear follows
Whisper's encoder. See NOTICE.
model-00003-of-00004.safetensors 5.0 GBmodel-00002-of-00004.safetensors 5.0 GBmodel-00001-of-00004.safetensors 5.0 GBmodel-00004-of-00004.safetensors 3.9 GBFunAudioLLM/Fun-Audio-Chat-8B