typhoon-whisper-large-v3-ctc

Word- and character-level timestamps for Thai: a 29M character-CTC head on the frozen typhoon-ai/typhoon-whisper-large-v3 encoder (pinned revision 748e8a4, recorded in config.json). One encoder pass gives transcription and alignment.

It is both an aligner and a compact ASR model: it force-aligns given text (a transcript or ASR hypothesis) to word/character timestamps, and its greedy CTC decode transcribes directly (--transcribe, no text needed).

Results

metric	value
greedy CER (ASR mode) on gigaspeech2-typhoon bench	6.72%
alignment jitter (solo-consistency, exact GT)	2.0 ms
onset p50 vs frame-exact TTS ground truth (thai-aligner-bench-dev)	25 ms (GT grid 25 ms)
onset bias (constant, correctable)	−20 ms (half-frame convention)
pause-instrument per-speaker r (thai-aligner-bench-dev)	0.988 (XLSR 0.961, MFA-Thai 0.767, MMS 0.752)
errors > 100 ms	0.09% (XLSR 0.20%)
word-onset p50 on human-annotated interviews (C-COST, data)	48.3 ms (constant +47 ms char-CTC-vs-MAUS convention offset; bias-discounted jitter 20.3 ms)

Trained on 1,000 h of GigaSpeech2-th.

Usage

pip install torch torchaudio transformers safetensors huggingface_hub soundfile pythainlp
python standalone_align.py --head typhoon-ai/typhoon-whisper-large-v3-ctc \
    --audio clip.wav --text "ข้อความ" --words

# ASR mode (greedy CTC decode)
python standalone_align.py --head typhoon-ai/typhoon-whisper-large-v3-ctc --audio clip.wav --transcribe

standalone_align.py (bundled) has no other dependencies; the frozen encoder downloads automatically at the pinned revision. Files: head.safetensors, config.json (architecture + encoder id/revision), ctc_vocab.json (80-symbol Thai char vocab v1.0.0).

How it works

Frozen Whisper encoder (50 Hz) → 29M Transformer head → per-frame character posteriors → CTC forced alignment against the given text → character spans on a 20 ms grid; words grouped with pythainlp newmm or caller-supplied segmentation.

Reference

Typhoon ASR: see the Typhoon ASR Real-time technical report.

Acknowledgements

We thank Kunat Pipatanakul and Potsawee Manakul (wayu-ai) for their valuable feedback.