TontaubeV1.1-int4

TontaubeV1.1-int4 is the 4-bit release of TontaubeV1.1 and the default model of the Tontaube inference server. It is a 1.8 GB download. On NVIDIA GPUs with bf16 support (Ampere or newer) it loads in about 4 GiB of GPU memory and peaks at about 6.5 GiB; it also runs on x86-64 Linux CPUs with AVX2, FMA, and F16C and a C++ compiler.

TontaubeV1.1 is a multilingual text-to-speech model designed for expressive voice cloning, long-form generation, and low-latency streaming. English and German are its primary languages; Spanish, French, Italian, Dutch, and Portuguese are also supported. Compared with TontaubeV1 it adds five style sliders (speaking rate, pauses, intonation, pitch, and crisp articulation), a VibeVoice adapter that refines the decoded audio, and a 4-bit version that runs on small GPUs and CPUs.

The release contains four causal codebook predictors. CB0 generates semantic audio and duration from text, while progressively smaller CB1--CB3 models add acoustic detail. Spoken text uses character-level tokenization, with compact BPE-encoded language and speaking-style controls. The public styles are audiobook, conversational, and agentic.

Technical report (TontaubeV1) · Try Tontaube online · Inference repository

Quick start

These checkpoints require the Tontaube inference server and its lite backend; they are not drop-in AutoModelForCausalLM checkpoints. Use the matching release of the inference repository:

git clone --branch v1.1.0 --depth 1 \
  https://github.com/craitech/tontaube.git
cd tontaube
uv run --python 3.12 --frozen tontaube serve

The same command runs on a machine without a GPU (or with LITE_DEVICE=cpu).

Once the API is ready, generate a WAV with the bundled English voice from a second terminal:

curl --silent --show-error --fail http://127.0.0.1:8080/predict \
  -H 'content-type: application/json' \
  -d '{"text":"Welcome to Tontaube.","language":"english","tag":"conversational","format":"wav"}' \
| python3 -c 'import base64,json,sys; sys.stdout.buffer.write(base64.b64decode(json.load(sys.stdin)["audio_b64"]))' \
> welcome.wav

Clone a voice by sending about 5--60 seconds of clean, single-speaker speech with the request:

VOICE_FILE=/path/to/reference.wav
python3 -c 'import base64,json,sys; print(json.dumps({"text":"This voice was cloned from a local reference.","language":"english","tag":"conversational","voice_audio_b64":base64.b64encode(open(sys.argv[1],"rb").read()).decode(),"format":"wav"}))' "$VOICE_FILE" \
| curl --silent --show-error --fail http://127.0.0.1:8080/predict \
  -H 'content-type: application/json' \
  --data-binary @- \
| python3 -c 'import base64,json,sys; sys.stdout.buffer.write(base64.b64decode(json.load(sys.stdin)["audio_b64"]))' \
> cloned.wav

Very quiet or very loud references are brought to a normal level before encoding.

The server downloads this model and its pinned runtime dependencies into local caches on first use. Follow the inference repository's installation instructions for system requirements, the browser interface, and Docker. The optional English verbalizer is available separately from TontaubeAI/TontaubeV1-Verbalizer and is downloaded only when enabled.

The default semantic sampling temperature is 0.7, with frequency penalty 0.1. A lower temperature generally favors text correctness, while a higher one can produce more varied and expressive delivery.

Style sliders

TontaubeV1.1 has five style sliders. Pass them as controls, one step per slider from 0 (low end) to the highest step; a slider left out keeps the model's own choice:

legato is one-sided: step 2 is the model's normal articulation and the crisp steps 0 and 1 also shorten pauses somewhat.

curl --silent --show-error --fail http://127.0.0.1:8080/predict \
  -H 'content-type: application/json' \
  -d '{"text":"Take it slowly.","controls":{"rate":1,"pause":3},"format":"wav"}' \
| python3 -c 'import base64,json,sys; sys.stdout.buffer.write(base64.b64decode(json.load(sys.stdin)["audio_b64"]))' \
> slow.wav

GET /controls on the server lists the sliders of the loaded model.

Examples

These MP3 samples were generated with TontaubeV1.1-int4 and normalized to a consistent speech listening level.

Audiobook

English 1

Download MP3

English 2

Download MP3

English 3

Download MP3

Agentic

English 1

Download MP3

English 2

Download MP3

Quantization

The transformer layers of all four predictors are GPTQ-quantized to 4 bits (groups of 64). The last decoder layer's MLP output projection and the audio head use 8 bits (groups of 128), the audio-token embeddings 8 bits per row, and the normalization weights and the text-embedding rows the input can reach stay in bf16. The full-precision weights are in TontaubeV1.1.

In our tests, the 4-bit weights stayed close to the full-precision model in accuracy and naturalness.

Speed

On an NVIDIA GeForce RTX 5090, streaming reaches a median of 192 ms to first audio, and synthesis runs at about 13 times real time for one request and 21--23 times real time across batched requests (32 active sequences per codebook). On a 12-core desktop CPU with dual-channel DDR4, synthesis runs at about real time; long streams can pause between chunks there.

Repository contents

TontaubeV1.1-int4/
├── cb0/                   (+ controls.json, cb0_ctc.pt)
├── cb1/
├── cb2/
├── cb3/
├── int4/
└── vibevoice_adapter.pt

int4/ holds the quantized layers of the four predictors; each codebook directory holds the remaining weights, runtime configuration, and tokenizer files. cb0/cb0_ctc.pt lets the server regenerate English pieces that skipped text, and vibevoice_adapter.pt refines VibeVoice's acoustic latent before decoding. The repository does not redistribute DualCodec, W2V-BERT, or VibeVoice weights. The inference setup downloads those components from their upstream repositories at pinned revisions.

Evaluation (TontaubeV1)

The results below were measured with TontaubeV1, the previous release, not with TontaubeV1.1.

The LLM-as-a-judge pairwise audiobook-reading benchmark contains 400 fixed English passages of 250--500 characters. For each passage, Gemini 3.1 Pro Preview judges the same output pair twice, once in each presentation order, on prosody and word-by-word correctness. A TontaubeV1 preference, tie, or comparator preference scores 1, 0.5, or 0; the figure reports the mean over all 800 order-balanced judgments. A score of 50% denotes parity. Whiskers show 95% passage-cluster bootstrap intervals; labels beneath the points give tie rates across individual judge calls.

Exact preference scores

TontaubeV1 uses semantic sampling temperature 0.55 and acoustic temperature zero in these comparisons. Each waveform is independently normalized to an average level of -20 dBFS before judging. The technical report gives the complete judge instructions and further methodological limitations.

Fish Audio uses the same frozen cloning reference as TontaubeV1. ElevenLabs, Gradium, and Cartesia instead use fixed provider voices while TontaubeV1 uses the cloning reference. Although voice identity and timbre are excluded from the rubric, prosody is not fully separable from the reference; this asymmetry may favor TontaubeV1 in those three comparisons.

This automated evaluation measures English reading prosody and text correctness. It does not establish voice similarity, overall sound quality, multilingual quality, long-form continuity, or streaming quality.

Limitations and responsible use

Autoregressive generation can omit, repeat, or alter text and can terminate too early or too late. Reference conditioning may reproduce incidental recording properties or imperfectly preserve identity. Quality varies by language, speaker, style, and text domain. The optional verbalizer can normalize incorrectly or alter wording. Audiobook generation may be more reliable than conversational or agentic generation.

English and German are the primary languages. German phoneme realization can be imperfect, and Spanish, French, Italian, Dutch, and Portuguese have not been evaluated by native speakers; no comparative quality claims are made for those languages.

Voice cloning can enable impersonation, fraud, non-consensual synthesis, and misleading media. Obtain permission for reference voices and apply appropriate authentication, rate limiting, disclosure, logging, and abuse-response measures in deployed systems. Generated speech must not be treated as verified identity or factual evidence.

License and attribution

TontaubeV1.1 is distributed under the Tontaube Community Model License 1.0, which is not an open-source license. The license applies to the Tontaube model materials identified by the release manifest (COVERED_FILES.md); third-party material remains subject to its respective terms. See Third-Party Notices.

The license names TontaubeV1, the model line this release belongs to; it covers the TontaubeV1.1 files listed in the manifest.

TontaubeV1.1 builds on Qwen3 and uses the DualCodec speech representation. The release inference stack also uses components from VibeVoice, W2V-BERT, PyTorch, Transformers, and vLLM. Their respective licenses and notices remain applicable.

The generated audio examples in assets/showcase/ are provided as demonstration material and are not part of the model materials covered by the Tontaube Community Model License 1.0.