Silencio

Small zero-shot text-to-speech that speaks in any voice.

Silencio turns English text into 16 kHz speech in a voice it has never heard before. A few seconds of reference audio are enough. All 33.6M parameters fit in one 134 MB model.safetensors. It renders audio in 120 ms chunks and runs about 9x faster than real time on one CPU thread of an Apple M4.

The same weights also do voice conversion: give it speech instead of text and it says the same words in another voice.

Silencio is built on two open models. The text front-end comes from Paradee-8M v1.1 by Sahil Mahendrakar, itself distilled from Kokoro-82M by hexgrad. The voice renderer and vocoder come from MeanVC2 by ASLP-lab. A new 3.4M-parameter content adapter connects them, so text goes straight into the renderer with no intermediate audio.

Model details

• Model type: zero-shot text-to-speech. A text encoder predicts phoneme durations, pitch and phonetic features. A ConvNeXt adapter turns them into content tokens at 25 Hz. A mean-flow DiT renders a mel spectrogram from those tokens and a 256-d voice embedding in 3 steps. Vocos then makes the waveform.
• Parameters: 33.6M (text encoder 4.2M, adapter 3.4M, renderer 17.7M, vocoder 8.3M)
• Weights files: model.safetensors, 134 MB (fp32), and model.int8.safetensors, 34 MB (int8, practically the same quality)
• Language: English (American pronunciation)
• Voices: zero-shot, from a 5–15 s reference clip. 8 ready-made voices are included.
• Output: mono, 16 kHz
• License: Apache 2.0

How it fits together

text ─► misaki G2P ─► text encoder ─► content adapter ─► DiT renderer ─► Vocos ─► 16 kHz audio
        (phonemes)    durations, F0,   content tokens     mean flow,        ▲
                      features @40 Hz  @25 Hz             3 steps           │
                                                            ▲               │
                                          voice embedding ──┘ (256-d)       │
speech ─► Fast-U2++ streaming ASR encoder ─► content tokens ─┘  (voice conversion)

The text encoder predicts how long each phoneme lasts and how the pitch moves. The renderer only decides how it sounds in the chosen voice. So timing and intonation come from the text, and timbre comes from the voice embedding.

Quickstart

Try it in Colab

Open the demo notebook and run all cells. It opens a Gradio demo with text to speech, voice cloning from the microphone or a file, and voice conversion.

Install

hf download BlueRoseLabs/Silencio --local-dir Silencio
cd Silencio
pip install -r requirements.txt
pip install --no-deps --ignore-requires-python misaki==0.9.4   # G2P; declares python<3.13 but runs on 3.13

Python

from silencio import SilencioTTS

tts = SilencioTTS.from_pretrained("BlueRoseLabs/Silencio")   # or the local folder
tts.save("speech.wav", "Hello from Silencio.", voice="librispeech_1320")

# Streaming: one chunk per sentence, float32 at 16 kHz
for pcm in tts.stream("Speech can start playing right away. The next sentence follows.", voice="librispeech_2094"):
    ...

For the 34 MB int8 weights, pass quant="int8" to from_pretrained. They are dequantized on load, so speed and memory use are the same as fp32.

generate returns a numpy array, and speed (default 1.0) changes the speaking rate. On first use misaki downloads spaCy's small English model (about 13 MB).

Any voice from a short clip

import soundfile as sf
from silencio.enroll import enroll        # pip install s3prl

voice = enroll(*sf.read("reference.wav"))  # 256-d embedding; save it with torch.save and reuse it
tts.save("cloned.wav", "This is my voice, read by Silencio.", voice=voice)

The voice encoder is WavLM-Large with ECAPA-TDNN, the same one MeanVC2 was trained with. It is large (about 1.3 GB) and only needed to add new voices, so it is not part of model.safetensors. It is downloaded on first use. A saved embedding needs nothing extra.

Voice conversion

wav, sr = sf.read("source.wav")
sf.write("converted.wav", tts.convert(wav, sr, voice="librispeech_2094"), 16000)

For speech input, the content tokens come from MeanVC2's streaming Fast-U2++ encoder. It is downloaded from ASLP-lab/MeanVC2 on first use and runs on CPU.

Voices

voices.safetensors holds 8 voices enrolled from LibriSpeech test-clean readers. A reader's voice is called librispeech_<speaker id>. The reference clip used for each one is included as ref_<id>.wav.

Samples

Five held-out sentences, read by Silencio in three voices. The last column is Paradee-8M v1.1, whose text encoder Silencio uses, in its single voice, for comparison. The sentences are in samples/sentences.txt.

Voice conversion: source (LibriSpeech 1320) → converted to librispeech_2094. Whisper transcribes both identically, and the speaker similarity to the target voice is 0.83.

Results

All numbers are on Paradee's 200 held-out WikiText sentences, scored the way the Paradee card scores them. Silencio speaks them in its 8 voices in turn.

• UTMOS is a neural network trained on human ratings. It predicts how natural a clip sounds, on a scale from 1 to 5 (SpeechMOS utmos22_strong).
• WER is the share of words that Whisper (base) transcribes wrongly.
• Speaker similarity is the cosine similarity between the voice of the output and the voice of the reference clip, measured with MeanVC2's WavLM/ECAPA encoder. 1.0 is the same embedding.
• Speed is how many times faster than real time the model runs, counting text to waveform, in PyTorch fp32. CPU speed is measured with one thread, on an Apple M4 (one two-sentence paragraph) and on the Colab host (10 held-out sentences).

Paradee was scored by the same script, in PyTorch, which is why its WER differs a little from the 5.9% on its own card.

Paradee speaks one studio voice at 24 kHz. Silencio speaks voices taken from audiobook recordings, and it does so at 16 kHz. Its UTMOS therefore also reflects the recording quality of the reference voices, not only the model.

How it was made

Only the adapter was trained. Everything else is reused frozen from the two base models.

1. Content space. MeanVC2's renderer takes content tokens from a streaming ASR encoder (256-d at 25 Hz) and a speaker embedding. Content tokens carry what is said and how, but not who says it.
2. Targets. Paradee read 6,000 WikiText sentences at speeds 0.9, 1.0 and 1.1 (about 6 hours). For each one, Paradee's internal text-side features were kept, and MeanVC2's ASR encoder was run on Paradee's audio to get the content tokens that audio would produce.
3. Adapter. A ConvNeXt network (4 blocks at 40 Hz, resampling to 25 Hz, then 4 more blocks) learns to map the text features, pitch and loudness to those content tokens. It uses an L1 plus cosine loss. Training took 15,000 steps at batch 32, about 22 minutes on one T4.
4. One model. The text encoder, adapter, renderer (EMA weights) and vocoder were put into a single model.safetensors. Weight norm was folded into the weights, and the Vocos TorchScript was converted to plain PyTorch. The output of the merged model matches the original chained pipeline to within 7e-4.

In speech-to-speech mode the original ASR encoder takes the adapter's place, so voice conversion works exactly as in MeanVC2.

Files

Limitations

• English only, with American pronunciation. Numbers, abbreviations and rare words are pronounced only as well as misaki handles them.
• Output is 16 kHz, which is the rate of MeanVC2's vocoder. That is enough for voice assistants and audiobooks, but not for full-band audio.
• Prosody comes from Paradee's single training voice, so every voice speaks with similar timing and intonation. Speaker similarity is lower than in voice conversion, where timing comes from the source speaker.
• Long text is split into sentences that are spoken one after another, so prosody does not carry across sentences.

Responsible use

Do not use Silencio to impersonate a real person, to deceive listeners about who is speaking, or to clone or convert a voice without the speaker's consent. Tell listeners when speech is synthetic where that matters to them.

License

The weights and code in this repository are released under Apache 2.0, the license of every model they are built from:

• Paradee-8M v1.1 by Sahil Mahendrakar (Apache 2.0): text encoder.
• Kokoro-82M by hexgrad (Apache 2.0): the architecture and teacher behind Paradee, and the misaki G2P.
• MeanVC2 by ASLP-lab (Apache 2.0): DiT renderer, Vocos vocoder, Fast-U2++ encoder and the speaker-encoder code.
• Vocos by Gemelo AI (MIT): vocoder architecture.

The voice encoder used by enroll (WavLM-Large from UniSpeech, with its ECAPA-TDNN speaker-verification fine-tune) is not redistributed here. It is downloaded at runtime under its own license. The bundled voices come from LibriSpeech (CC BY 4.0) readers, whose recordings are from the public-domain LibriVox project.

Citation

@misc{silencio2026,
  title  = {Silencio: Zero-Shot Text-to-Speech by Connecting a Distilled Text Encoder to a Streaming Voice Renderer},
  author = {{BlueRose Labs}},
  year   = {2026},
  url    = {https://huggingface.co/BlueRoseLabs/Silencio}
}

@misc{mahendrakar2026paradee,
  title  = {Paradee: Distilling Kokoro-82M into an 8M-Parameter Single-Voice Text-to-Speech Model},
  author = {Mahendrakar, Sahil},
  year   = {2026},
  url    = {https://huggingface.co/sahilmahendrakar/Paradee-8M-v1.0}
}

MeanVC2: github.com/ASLP-lab/MeanVC2.