typhoon-whisper-medium-ctc
Word- and character-level timestamps for Thai: a 29M character-CTC head on
the frozen typhoon-ai/typhoon-whisper-medium
encoder (pinned revision 945a87c, recorded in config.json). One encoder
pass gives transcription and alignment.
It is both an aligner and a compact ASR model: it force-aligns given
text (a transcript or ASR hypothesis) to word/character timestamps, and its
greedy CTC decode transcribes directly (--transcribe, no text needed).
Results
| metric | value |
|---|---|
| greedy CER (ASR mode) on gigaspeech2-typhoon bench | 6.86% |
| alignment jitter (solo-consistency, exact GT) | 2.0 ms |
| onset p50 vs frame-exact TTS ground truth (thai-aligner-bench-dev) | 20 ms (GT grid 25 ms) |
| onset bias (constant, correctable) | −15 ms (half-frame convention) |
| pause-instrument per-speaker r (thai-aligner-bench-dev) | 0.979 (XLSR 0.961, MFA-Thai 0.767, MMS 0.752) |
| errors > 100 ms | 0.09% (XLSR 0.20%) |
| word-onset p50 on human-annotated interviews (C-COST, data) | 54.0 ms (constant +53 ms char-CTC-vs-MAUS convention offset; bias-discounted jitter 21.3 ms) |
Trained on 1,000 h of GigaSpeech2-th.
Usage
pip install torch torchaudio transformers safetensors huggingface_hub soundfile pythainlp
python standalone_align.py --head typhoon-ai/typhoon-whisper-medium-ctc \
--audio clip.wav --text "ข้อความ" --words
# ASR mode (greedy CTC decode)
python standalone_align.py --head typhoon-ai/typhoon-whisper-medium-ctc --audio clip.wav --transcribe
standalone_align.py (bundled) has no other dependencies; the frozen encoder
downloads automatically at the pinned revision. Files: head.safetensors,
config.json (architecture + encoder id/revision), ctc_vocab.json
(80-symbol Thai char vocab v1.0.0).
How it works
Frozen Whisper encoder (50 Hz) → 29M Transformer head → per-frame character
posteriors → CTC forced alignment against the given text → character spans on
a 20 ms grid; words grouped with pythainlp newmm or caller-supplied
segmentation.
Reference
Typhoon ASR: see the Typhoon ASR Real-time technical report.
Acknowledgements
We thank Kunat Pipatanakul and Potsawee Manakul (wayu-ai) for their valuable feedback.