No GPU? Try it through the hosted API: sign up and get 3 hours of free voice chat (OpenAI Realtime-compatible, docs).
Version 1.2.0 · changelog · code: mac/ · CUDA build: Sophon-Realtime-32B-Flash-4bit · 中文:README_CN.md
The full-duplex speech model Sophon-Realtime-32B-Flash-4bit, running locally on an Apple silicon Mac with MLX. You talk, it decides by itself when you have finished, answers with its own voice and sound tags, stops when you interrupt, and carries on when you just say "mm-hmm". Hearing, turn-taking and speaking are all done by the model on the Mac's GPU: no cloud, and no speech-to-text step in between.
The weights are the Flash-4bit weights repacked losslessly for MLX: the 4-bit values, scales, audio tower and voice stage are bit-identical to the CUDA build.
git clone https://github.com/adventists-ai/sophon-realtime && cd sophon-realtime
bash mac/install.sh # Python environment + this repo's weights, into ~/sophon-mac
~/sophon-mac/sophon # talk in Chinese
~/sophon-mac/sophon --lang en # talk in English
Options: --no-aec (headset, no echo cancellation), --in / --out (audio devices, --list shows them), --voice.
| You stop talking → it starts speaking | 0.9–1.1 s when the model is sure you have finished; up to about 2.5 s when it waits to see whether you go on |
| Long replies | real time, no stalls |
| You interrupt | it stops within about 0.15 s |
| Model load | about 10 s |
The CUDA server answers in about 0.5 s. On a Mac most of the time goes into the turn decision: when you pause, the model listens to your whole utterance once on the Mac's GPU. Chips with more GPU cores are faster.
To process your audio only once, the turn question is asked inside the reply's own context (system prompt, conversation, your audio, then a short "has the user finished? A / B" question); if the answer is yes, the reply continues from the same computation. The check schedule and thresholds are those of the server (0.2 s to 2 s of silence, then a fallback reply). Accuracy on the Smart Turn test set (400 clips per language):
| English acc. / AUC | Chinese acc. / AUC | |
|---|---|---|
| Server turn question | 92.5 / 0.975 | 94.0 / 0.983 |
| Mac turn question | 90.5 / 0.960 | 94.3 / 0.977 |
Right after you interrupt, the full four-option question is used instead, so that a short back-channel ("mm-hmm", "yeah") makes it resume the cut reply.
WebRTC AEC3 removes the reply's own sound from the microphone. In a simulated room (speaker sound back at the microphone at −6 dB and 0 dB, 6 s of reply), the reply never interrupted itself with echo cancellation on (3 / 3 runs) and did every time with it off; a real interruption was still detected within about 0.12 s.
| Folder | What |
|---|---|
thinker_mlx/ |
the 4-bit thinker for mlx_lm (same weights as Flash-4bit) |
ear/ |
the thinker's audio tower and projector |
talker/ |
the voice stage (same as Flash) |
voices/ |
built-in voices |
License: code Apache-2.0; model weights Apache-2.0, see NOTICE for sources.
talker/model.safetensors 3.9 GBthinker_mlx/model-00000.safetensors 3.3 GBthinker_mlx/model-00004.safetensors 1.1 GBthinker_mlx/model-00010.safetensors 1.1 GBthinker_mlx/model-00007.safetensors 1.1 GBthinker_mlx/model-00013.safetensors 1.1 GBthinker_mlx/model-00014.safetensors 1.1 GBthinker_mlx/model-00011.safetensors 1.1 GBthinker_mlx/model-00005.safetensors 1.1 GBthinker_mlx/model-00008.safetensors 1.1 GBthinker_mlx/model-00009.safetensors 1.1 GBthinker_mlx/model-00012.safetensors 1.1 GBthinker_mlx/model-00002.safetensors 1.1 GBthinker_mlx/model-00003.safetensors 1.1 GBthinker_mlx/model-00006.safetensors 1.1 GBthinker_mlx/model-00001.safetensors 1.1 GBthinker_mlx/model-00015.safetensors 708 MBtalker/speech_tokenizer/model.safetensors 682 MBear/ear.safetensors 405 MB