Thaki-SyntheticVoice-KO-2B-v0.1

A VoxCPM2 fine-tune on a ThakiCloud-designed synthetic Korean speaker. With the bundled fixed reference clip it performs on par with VoxCPM2 Base + the same reference. Without a reference it is clearly better than the base model in pronunciation accuracy and naturalness, but speaker consistency across utterances is still limited.

🇰🇷 한국어 카드: MODEL_CARD.ko.md

Engineering blog: A Korean Voice Built Without a Real Person (한국어: 블로그)

⚠️ What this model is not. It is not a better TTS than VoxCPM2 Base+Ref, and it does not guarantee an identical voice on every generation without a reference. It was evaluated only on Korean, clean synthetic conditions, one female voice, no emotion control. Read §4 before deploying.

0. Quick start

pip install -U voxcpm   # the pinned commit we used: OpenBMB/VoxCPM@cce58a5b

Recommended: with the bundled reference (this is the configuration that was evaluated as "on par with Base+Ref").

import soundfile as sf
from voxcpm import VoxCPM

model = VoxCPM.from_pretrained("ThakiCloud/Thaki-SyntheticVoice-KO-2B-v0.1", load_denoiser=False)

REF_WAV  = "reference/anchor_F1_16k.wav"   # bundled, 8.3 s, 16 kHz
REF_TEXT = "안녕하세요. 저는 타키클라우드의 음성 비서입니다. 오늘도 차분하고 또렷한 목소리로 필요한 정보를 안내해 드리겠습니다."

wav = model.generate(
    text="요청하신 작업은 오늘 오후 세 시 삼십 분에 완료될 예정입니다.",
    prompt_wav_path=REF_WAV, prompt_text=REF_TEXT,
    cfg_value=2.0, inference_timesteps=10, seed=0,
)
sf.write("out.wav", wav, model.tts_model.sample_rate)   # 48 kHz

Experimental: without a reference. Produces a voice close to F1 with good naturalness, but the timbre varies between utterances (§2.2). Do not rely on it for a fixed brand voice yet.

wav = model.generate(text="안녕하세요. 타키클라우드 음성 안내입니다.", cfg_value=2.0, inference_timesteps=10, seed=0)

Input text convention. Write numbers, units and symbols the way they should be spoken (Korean words), e.g. 3억 2,500만 원 → 삼억 이천오백만 원, 오후 3시 30분 → 오후 세 시 삼십 분, B200 8장 → 비 이백 여덟 장, 12% → 십이 퍼센트. The training transcripts follow this convention; Arabic digits in the input were not trained and may be read inconsistently.

Optional: version-string preprocessing (separate code, not part of the model). The weights in this repository are the v0.1 weights and do not normalize text by themselves. In a follow-up check, the one input type that was read wrongly in display form was version strings: v1.2.3 (the v read as a letter name) and 2.0 (a dropped "zero"). For that case we ship a small rule-based normalizer as plain Python files in this repository, under text_normalization/ (text_norm.py + korean_numbers.py, no dependencies). VoxCPM.from_pretrained does not load it; you call it yourself before generate.

import sys
from huggingface_hub import snapshot_download

d = snapshot_download("ThakiCloud/Thaki-SyntheticVoice-KO-2B-v0.1", allow_patterns=["text_normalization/*"])
sys.path.insert(0, f"{d}/text_normalization")
from text_norm import normalize

text = normalize("VoxCPM 2.0 버전과 v1.2.3-rc1을 비교합니다.")
# -> "VoxCPM 이 점 영 버전과 버전 일 점 이 점 삼 알씨 일을 비교합니다."
wav = model.generate(text=text, prompt_wav_path=REF_WAV, prompt_text=REF_TEXT,
                     cfg_value=2.0, inference_timesteps=10, seed=0)

It only rewrites version strings: v-prefixed numbers, dotted numbers with three or more parts, and two-part numbers right after a product name or before "버전/version". Dates (2026.10.26), IP addresses, URLs, e-mail addresses, file paths and decimals with units (3.5GB, 3.5배) are left as they are, and every other number still follows the input convention above. On our 1,200-sentence script it changed 34 sentences, all to exactly the hand-written spoken form, and left spoken-form input unchanged. It is not a general Korean text normalizer.

1. What was trained

1. Speaker design. Eight candidate voices (4 female, 4 male) were generated with VoxCPM2 Voice Design from text descriptions only (gender, age range, pitch, tone, pace). For each, a fixed 8.3 s anchor sentence and ten test sentences were synthesized and scored for naturalness (UTMOS), speaker consistency (WavLM-SV cosine to the anchor) and content (whisper CER). F1 was selected (UTMOS 3.46, cosine to anchor 0.988 mean / 0.976 min). The anchor file is the bundled reference (sha256 cb73d996b1b9f0017bcd428d00ea12fd9dd262b60a0111640f13f898906dfbad).
2. Script. 1,200 self-written Korean sentences (general, enterprise/tech, numbers/money/dates, alphanumerics/product names, long sentences, customer-service dialogue, hard pronunciation), each with a display form and a spoken form; split by template so that no template appears in two splits.
3. Synthesis. VoxCPM2 Base + the F1 reference synthesized the 1,000 training-split sentences (cfg 2.0, 10 steps). Each utterance was kept only if whisper-large-v3-turbo back-transcription matched the spoken text (CER ≤ 8 % after numeral-aware normalisation), WavLM-SV cosine to the anchor ≥ 0.95, UTMOS ≥ 2.9, no runaway/silence. 934 of 1,000 passed (80.8 min); 888 train / 46 validation utterances.
4. LoRA. r 32, α 32, lr 1e-4, batch 4 × grad-accum 4, 2 epochs (112 steps), 40 % of rows with a same-speaker reference clip. Validation loss 0.689 → 0.586. Then merged into the base (W += B·A·α/r, 192 modules) and saved as one checkpoint.

This is self-distillation: the training audio is VoxCPM2's own output. It cannot teach the model anything the base cannot already produce; what it can do is bias the model toward one voice. Interpret the results accordingly.

2. Results (our measurements; same hardware, decoding and scorer for both arms)

Conditions: 1× H100 NVL, bf16, cfg_value=2.0, inference_timesteps=10; CER = whisper-large-v3-turbo back-transcription vs. spoken text (Arabic-numeral output counts against both arms equally); speaker cosine = WavLM-base-plus-sv; naturalness = UTMOS22; RTF = synthesis time / audio length. Medians over seeds.

2.1 With the fixed F1 reference (200 held-out sentences × seeds 0/1/2)

Verdict: non-inferior. No meaningful advantage over Base+Ref in this configuration. (The un-merged LoRA path was 22 % slower, 0.176; merging removes that overhead.)

2.2 Without any reference (60 category-balanced held-out sentences × seeds 0/1/2)

For scale: with the reference, cosine to the anchor is 0.98. Reading: without a reference the base model picks a different random voice each time; this model pulls strongly toward F1 and sounds much more natural, but a minimum cosine of 0.40 means some utterances still come out in a noticeably different voice. Speaker locking without a reference is partially achieved, not solved.

3. Intended use

• Korean read-speech / assistant-style prompts in the F1 voice, with the bundled reference, where you want a ready-made synthetic voice instead of cloning a person's recording.
• Research on reference-free speaker consistency (the no-reference numbers above are the baseline to beat).

Not intended for: impersonating any real person; emotional/expressive speech; languages other than Korean; production deployment without your own evaluation.

4. Limitations

• Speaker consistency without a reference is limited (anchor cosine 0.79 mean, 0.40 min). Use the reference for a stable voice.
• No advantage over Base+Ref with a reference. If you already pass a reference clip, the base model gives the same quality.
• Single female voice, neutral style. No emotion, speed or style control was trained or evaluated.
• Synthetic-only training data (self-distillation). Pronunciation errors of the base model can be inherited; we removed utterances the filters caught, and a human listening check of 30 number/English-alphanumeric outputs from this checkpoint passed 30/30, but the training data was not exhaustively hand-verified.
• Numerals must be written in Korean words (see §0). Arabic digits were not in the training transcripts.
• Evaluation scope: Korean, clean synthetic audio, whisper-based CER (which penalises both arms equally for numeral formatting), UTMOS22 (not calibrated for Korean), one GPU type. No listening test with external raters.
• RTF numbers are from an H100; expect different absolute values elsewhere.

5. Rights and data

• The speaker is synthetic, created from a text description with VoxCPM2 Voice Design. No real person's recording was used as a reference at any stage, and no celebrity, voice actor or character was named in the description. This is a factual statement about the process; it is not a legal opinion that no third-party right can be affected.
• No AI Hub data and no human voice-actor recordings were used for this model (unlike our ASR releases).
• Training audio (934 synthesized utterances) is not included in this repository. The 8.3 s synthetic reference clip is included so that the recommended configuration is reproducible.
• Base model openbmb/VoxCPM2: Apache-2.0. These merged weights are a modified version of it, released under Apache-2.0; see LICENSE and NOTICE for the license text, attribution and a summary of our changes.
• On generated audio: the VoxCPM maintainers stated (OpenBMB/VoxCPM #407, #409, 2026-10-12) that Voice Design output may be saved, reused as a reference, published, redistributed and used commercially, subject to applicable law and third-party rights; they noted this is their stance, not legal advice. Their statement does not specifically address training on generated audio, which is how this model was made.
• Please do not use the model to imitate an identifiable person, and tell listeners when speech they hear is AI-generated, as many jurisdictions require.

6. Reproducibility

• Base: openbmb/VoxCPM2 (S3 mirror of the Hub checkpoint; voxcpm at commit cce58a5b).
• LoRA: official train_voxcpm_finetune.py with the config in §1; merge: W += B·A·α/r over the 192 LoRA linear modules; the merged state dict was loaded into a plain base object with 0 missing / 0 unexpected keys.
• Decoding for every number here: cfg_value=2.0, inference_timesteps=10, seeds 0/1/2.
• SHA-256 of every file was checked before and after upload (see commit message / ledger).

7. Citation

@misc{thakicloud2026syntheticvoice,
  title  = {Thaki-SyntheticVoice-KO-2B-v0.1: VoxCPM2 fine-tuned on a designed synthetic Korean speaker},
  author = {ThakiCloud},
  year   = {2026},
  url    = {https://huggingface.co/ThakiCloud/Thaki-SyntheticVoice-KO-2B-v0.1}
}

Base model: OpenBMB, VoxCPM2: Tokenizer-Free TTS for Multilingual Speech Generation, Creative Voice Design, and True-to-Life Cloning.