WAXAL VoxCPM2 — 28-Language African TTS

A full fine-tune of VoxCPM2 (2B params, tokenizer-free, diffusion-AR TTS with voice cloning) on 28 African languages, produced by the WAXAL project at LyngualLabs.

This is a research checkpoint. It is not production-polished — see Known Limitations below before using it for anything beyond experimentation.

Languages

VoxCPM2 has no built-in mechanism for specifying language at inference — the base model infers pronunciation purely from text orthography. We added one: a dedicated <lang:XX> tag per language, registered as a real token in the tokenizer and resized into the text embedding table (mean-initialized, then learned during fine-tuning). The tag is mandatory — always prepend it to the text you want synthesized, e.g.:

<lang:yo> Ẹ káàárọ̀! Báwo ni oorun rí lálẹ́ àná?

Source data: the AfroNet speech collection (Professor/<language>-speech-data), 7,424 raw hours total.

Training

• Base: openbmb/VoxCPM2, with 28 new <lang:XX> tokens added and the text embedding table resized (mean-initialized rows) before any fine-tuning started — this is an independent run from scratch, not built on top of any earlier narrower fine-tune.
• Mode: full fine-tuning (all parameters trainable except the frozen AudioVAE codec).
• Data balancing: the 28 languages range 35x in size (1038.7h Yoruba vs. 29.7h Acholi). Naive concatenation would starve the smallest languages; full equalization would cause heavy repetition/overfitting on them. We used sqrt-temperature resampling (weight ∝ √hours, renormalized to preserve total volume) — a standard middle-ground technique from multilingual ASR/TTS training. This downsamples the biggest languages to ~58–97% of their raw hours and upsamples the smallest to ~1.4–3.4x, via a combination of subsampling and controlled repetition.
• Scale: 2,929,563 training rows (clips) after resampling, 9,450 held-out validation rows.
• Steps: 44,000 steps (~0.99 epoch over the resampled training set), batch size 3 × grad-accum 11 × 2 GPUs (effective batch 66).
• Hardware: 2× NVIDIA H200, via torchrun DDP.
• Learning rate: 1e-5, 100 warmup steps.

Usage

from huggingface_hub import snapshot_download
from voxcpm import VoxCPM
import soundfile as sf

model_dir = snapshot_download("waxal-benchmarking/voxcpm2-multilang28-full")
model = VoxCPM.from_pretrained(model_dir, load_denoiser=False, optimize=False)

wav = model.generate(
    text="<lang:yo> Ẹ káàárọ̀! Báwo ni oorun rí lálẹ́ àná?",
    cfg_value=2.0,
    inference_timesteps=22,
)
sf.write("out.wav", wav, model.tts_model.sample_rate)

optimize=False is required — torch.compile (the default optimization path) conflicts with some inference environments (e.g. HF ZeroGPU Spaces); disabling it costs some speed but avoids crashes.

Voice cloning works the same way as base VoxCPM2 — pass prompt_wav_path and prompt_text (the literal transcript of what's spoken in the reference clip) alongside the tagged text.

Try it live: WAXAL VoxCPM Multilingual Demo Space.

Evaluation

We built an automated evaluation pipeline rather than relying on casual listening, using:

• ASR-based CER (Character Error Rate) via Meta's Omnilingual ASR (omniASR_CTC_7B, ~1,600-language coverage, verified against this model's own supported-language table — not assumed), comparing transcriptions of generated speech against the ground-truth input text.
• The same ASR run on real human recordings of the same held-out text, as a per-language noise floor — several of these languages get nontrivial CER even on real speech (e.g. Fante in an earlier AfroNet pilot scored CER 0.49–0.57 on real recordings), so a raw TTS CER number is meaningless without that baseline.
• DNSMOS (no-reference naturalness proxy). Caveat: DNSMOS is trained on English/high-resource-language listener ratings, so treat these scores as relative (comparing our own languages/checkpoints) rather than absolute.

Pilot results (5 held-out validation clips per language, no reference voice):

The non-terminating-generation issue doesn't cleanly correlate with raw data size — Swahili (one of the largest languages here) is affected while several much smaller languages (Hausa, Acholi, Dagbani, Wolof) show none. It's an open issue, not yet root-caused.

Known Limitations

• Voice-design and cross-lingual flexibility are likely reduced. Earlier narrow single-language full fine-tunes on this same base model showed that full-FT on read-speech-only data degrades the base model's text-prompted style/voice-design control and non-trained-language generation. This 28-language run is far more linguistically diverse than those single-language runs, which should help, but it is still exclusively plain narration-style audio — we have not verified how much voice-design capability survives.
• Non-terminating generation (see Evaluation) affects a meaningful minority of generations, worse for some languages (Malagasy, Sidama, Somali) than others.
• Translation/transcript quality varies by language in our own test tooling — some of the lowest-resource languages here (Wolaytta, Sidama in particular) have had machine-translated test sentences that looked suspect on inspection; this is a tooling caveat about our test sentences, not necessarily the model.
• Language coverage via Omnilingual ASR for evaluation purposes was spot-checked, not independently verified for linguistic accuracy of every transcription.

Related

• waxal-benchmarking/voxcpm2-multilang10-full — an earlier, independent fine-tune on just the 10 highest-resource languages from the same collection (Yoruba, Swahili, Kinyarwanda, Igbo, Hausa, Wolaytta, Dholuo, Amharic, Oromo, Sidama), trained the same way.
• AfroNet speech collection — the source training data.