TontaubeV1.1

TontaubeV1.1 is a multilingual text-to-speech model designed for expressive voice cloning, long-form generation, and low-latency streaming. English and German are its primary languages; Spanish, French, Italian, Dutch, and Portuguese are also supported. Compared with TontaubeV1 it adds five style sliders (speaking rate, pauses, intonation, pitch, and crisp articulation), a VibeVoice adapter that refines the decoded audio, and a 4-bit version that runs on small GPUs and CPUs.

The release contains four causal codebook predictors. CB0 generates semantic audio and duration from text, while progressively smaller CB1--CB3 models add acoustic detail. Spoken text uses character-level tokenization, with compact BPE-encoded language and speaking-style controls. The public styles are audiobook, conversational, and agentic.

This repository holds the full-precision (bf16) weights, served by the vLLM backend of the inference server and the source for quantized builds. For most uses, TontaubeV1.1-int4 is the better start: the inference server's default, a 1.8 GB instead of a 6.4 GB download, and it runs on small GPUs and on CPUs.

Technical report (TontaubeV1) · Try Tontaube online · Inference repository

Quick start

These checkpoints require the Tontaube inference server and its input layout; they are not drop-in AutoModelForCausalLM checkpoints. Use the matching release of the inference repository. The vLLM backend needs an NVIDIA GPU with at least 24 GiB of memory:

git clone --branch v1.1.0 --depth 1 \
  https://github.com/craitech/tontaube.git
cd tontaube
TTS_BACKEND=vllm uv run --python 3.12 --frozen tontaube serve

Once the API is ready, generate a WAV with the bundled English voice from a second terminal:

curl --silent --show-error --fail http://127.0.0.1:8080/predict \
  -H 'content-type: application/json' \
  -d '{"text":"Welcome to Tontaube.","language":"english","tag":"conversational","format":"wav"}' \
| python3 -c 'import base64,json,sys; sys.stdout.buffer.write(base64.b64decode(json.load(sys.stdin)["audio_b64"]))' \
> welcome.wav

Clone a voice by sending about 5--60 seconds of clean, single-speaker speech with the request:

VOICE_FILE=/path/to/reference.wav
python3 -c 'import base64,json,sys; print(json.dumps({"text":"This voice was cloned from a local reference.","language":"english","tag":"conversational","voice_audio_b64":base64.b64encode(open(sys.argv[1],"rb").read()).decode(),"format":"wav"}))' "$VOICE_FILE" \
| curl --silent --show-error --fail http://127.0.0.1:8080/predict \
  -H 'content-type: application/json' \
  --data-binary @- \
| python3 -c 'import base64,json,sys; sys.stdout.buffer.write(base64.b64decode(json.load(sys.stdin)["audio_b64"]))' \
> cloned.wav

Very quiet or very loud references are brought to a normal level before encoding.

The server downloads this model and its pinned runtime dependencies into local caches on first use; without TTS_BACKEND=vllm it downloads TontaubeV1.1-int4 instead. Follow the inference repository's installation instructions for system requirements, the browser interface, and Docker. The optional English verbalizer is available separately from TontaubeAI/TontaubeV1-Verbalizer and is downloaded only when enabled. To build your own quantized version (for example int8), use tontaube quantize; see the inference repository's lite backend section.

The default semantic sampling temperature is 0.7, with frequency penalty 0.1. A lower temperature generally favors text correctness, while a higher one can produce more varied and expressive delivery.

Style sliders

TontaubeV1.1 has five style sliders. Pass them as controls, one step per slider from 0 (low end) to the highest step; a slider left out keeps the model's own choice:

legato is one-sided: step 2 is the model's normal articulation and the crisp steps 0 and 1 also shorten pauses somewhat.

curl --silent --show-error --fail http://127.0.0.1:8080/predict \
  -H 'content-type: application/json' \
  -d '{"text":"Take it slowly.","controls":{"rate":1,"pause":3},"format":"wav"}' \
| python3 -c 'import base64,json,sys; sys.stdout.buffer.write(base64.b64decode(json.load(sys.stdin)["audio_b64"]))' \
> slow.wav

GET /controls on the server lists the sliders of the loaded model.

Capabilities

• Bundled synthetic voices and zero-shot cloning from up to 60 seconds of reference audio
• audiobook, conversational, and agentic speaking styles
• Five style sliders
• Bounded-context long-form generation
• WAV generation and low-latency MP3 or Opus streaming
• Optional English text verbalizer, distributed separately

Examples

These MP3 samples were generated with TontaubeV1.1-int4 and normalized to a consistent speech listening level.

Audiobook

English 1

Download MP3

English 2

Download MP3

English 3

Download MP3

Agentic

English 1

Download MP3

English 2

Download MP3

Model architecture

CB0 uses a Qwen3-1.7B-derived transformer trunk. CB1--CB3 use progressively shallower Qwen3-0.6B-derived trunks. Each predictor replaces the inherited language-model output layer with a two-layer audio-token head. The four output streams are decoded with DualCodec; the release inference path then uses the acoustic encoder and decoder from VibeVoice for continuous reconstruction and streaming. A small VibeVoice adapter (vibevoice_adapter.pt) refines VibeVoice's acoustic latent before decoding, and a recogniser on CB0's output (cb0/cb0_ctc.pt) lets the server retry English pieces that skipped text.

Repository contents

TontaubeV1.1/
├── cb0/                   (+ controls.json, cb0_ctc.pt)
├── cb1/
├── cb2/
├── cb3/
└── vibevoice_adapter.pt

Each codebook directory contains its safetensors weights, runtime configuration, and tokenizer files. The repository does not redistribute DualCodec, W2V-BERT, or VibeVoice weights. The inference setup downloads those components from their upstream repositories at pinned revisions.

Evaluation (TontaubeV1)

The results below were measured with TontaubeV1, the previous release, not with TontaubeV1.1.

The LLM-as-a-judge pairwise audiobook-reading benchmark contains 400 fixed English passages of 250--500 characters. For each passage, Gemini 3.1 Pro Preview judges the same output pair twice, once in each presentation order, on prosody and word-by-word correctness. A TontaubeV1 preference, tie, or comparator preference scores 1, 0.5, or 0; the figure reports the mean over all 800 order-balanced judgments. A score of 50% denotes parity. Whiskers show 95% passage-cluster bootstrap intervals; labels beneath the points give tie rates across individual judge calls.

Exact preference scores

TontaubeV1 uses semantic sampling temperature 0.55 and acoustic temperature zero in these comparisons. Each waveform is independently normalized to an average level of -20 dBFS before judging. The technical report gives the complete judge instructions and further methodological limitations.

Fish Audio uses the same frozen cloning reference as TontaubeV1. ElevenLabs, Gradium, and Cartesia instead use fixed provider voices while TontaubeV1 uses the cloning reference. Although voice identity and timbre are excluded from the rubric, prosody is not fully separable from the reference; this asymmetry may favor TontaubeV1 in those three comparisons.

This automated evaluation measures English reading prosody and text correctness. It does not establish voice similarity, overall sound quality, multilingual quality, long-form continuity, or streaming quality.

Limitations and responsible use

Autoregressive generation can omit, repeat, or alter text and can terminate too early or too late. Reference conditioning may reproduce incidental recording properties or imperfectly preserve identity. Quality varies by language, speaker, style, and text domain. The optional verbalizer can normalize incorrectly or alter wording. Audiobook generation may be more reliable than conversational or agentic generation.

English and German are the primary languages. German phoneme realization can be imperfect, and Spanish, French, Italian, Dutch, and Portuguese have not been evaluated by native speakers; no comparative quality claims are made for those languages.

Voice cloning can enable impersonation, fraud, non-consensual synthesis, and misleading media. Obtain permission for reference voices and apply appropriate authentication, rate limiting, disclosure, logging, and abuse-response measures in deployed systems. Generated speech must not be treated as verified identity or factual evidence.

License and attribution

TontaubeV1.1 is distributed under the Tontaube Community Model License 1.0, which is not an open-source license. The license applies to the Tontaube model materials identified by the release manifest (COVERED_FILES.md); third-party material remains subject to its respective terms. See Third-Party Notices.

The license names TontaubeV1, the model line this release belongs to; it covers the TontaubeV1.1 files listed in the manifest.

TontaubeV1.1 builds on Qwen3 and uses the DualCodec speech representation. The release inference stack also uses components from VibeVoice, W2V-BERT, PyTorch, Transformers, and vLLM. Their respective licenses and notices remain applicable.

The generated audio examples in assets/showcase/ are provided as demonstration material and are not part of the model materials covered by the Tontaube Community Model License 1.0.