Tha-ASR: Khmer + English speech recognition

Tha-ASR is a Whisper-small model fine-tuned to transcribe Khmer, English, and Khmer–English code-switched speech, where both languages are mixed in the same sentence. It writes Khmer in Khmer script and English in Latin script, with punctuation.

Compared with the previous version (v1), this release cuts the English word error rate from 11.1 % to 8.3 % and the code-switch character error rate from 14.1 % to 4.5 %, and is slightly better on Khmer.

Evaluation

All test sets are held out: no training utterance shares its text with a test or dev sentence.

How it was measured. Clean audio, beam search with 5 beams, batch size 4, max_new_tokens=440, and the language token set to each test set's language. Khmer is scored with CER after Unicode NFC normalisation, removing zero-width characters, and ignoring spaces and punctuation (Khmer has no reliable word spacing, so WER is not meaningful). English is scored with WER after lower-casing and removing punctuation. Lower is better.

Read these numbers with care:

• 537 of the 765 khm-asr-open test sentences also appear in the FLEURS Khmer test set, so the two Khmer results largely overlap.
• The code-switch set was built by the same data pipeline that produced the code-switch training data, not collected from natural conversations, and it is small. Expect higher error rates on spontaneous mixed-language speech.

How to use

pip install torch transformers librosa

Transcribe a clip (up to 30 s)

import librosa
import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor

model_id = "Thareah/Tha-ASR"
processor = WhisperProcessor.from_pretrained(model_id)
model = WhisperForConditionalGeneration.from_pretrained(model_id).eval()

audio, _ = librosa.load("speech.wav", sr=16000, mono=True)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
with torch.inference_mode():
    ids = model.generate(inputs.input_features, language="km", task="transcribe", num_beams=5, max_new_tokens=440)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])

Use language="en" for English.

Detect Khmer or English automatically

Whisper's built-in language detection chooses among 99 languages and can drift to Thai or Lao on Khmer speech. Comparing only the Khmer and English language tokens is more reliable:

def detect_km_en(features):
    tok = processor.tokenizer
    lang_ids = [tok.convert_tokens_to_ids("<|km|>"), tok.convert_tokens_to_ids("<|en|>")]
    start = torch.tensor([[tok.convert_tokens_to_ids("<|startoftranscript|>")]])
    with torch.inference_mode():
        logits = model(input_features=features, decoder_input_ids=start).logits[0, -1]
    return ("km", "en")[int(logits[lang_ids].argmax())]

language = detect_km_en(inputs.input_features)
ids = model.generate(inputs.input_features, language=language, task="transcribe", num_beams=5, max_new_tokens=440)

Long recordings

Cut long audio at pauses into pieces of at most 15 s, then transcribe the pieces as a batch. Khmer needs many tokens per second of speech, and Whisper's decoder holds only 448 tokens. Splitting at pauses also avoids the repetition loops that fixed-length chunking can cause. This example uses the Silero voice detector that comes with faster-whisper (pip install faster-whisper):

from faster_whisper.vad import VadOptions, get_speech_timestamps

audio, _ = librosa.load("meeting.mp3", sr=16000, mono=True)
spans = get_speech_timestamps(audio, VadOptions(max_speech_duration_s=15, min_silence_duration_ms=300, speech_pad_ms=200))
pieces = [audio[s["start"]:s["end"]] for s in spans]
features = processor(pieces, sampling_rate=16000, return_tensors="pt").input_features
with torch.inference_mode():
    ids = model.generate(features, language="km", task="transcribe", num_beams=5, max_new_tokens=440)
for s, text in zip(spans, processor.batch_decode(ids, skip_special_tokens=True)):
    print(f"[{s['start'] / 16000:7.1f}s - {s['end'] / 16000:7.1f}s] {text.strip()}")

For many pieces, run them in batches of 8–16 to limit memory. On a GPU, load the model with torch_dtype=torch.float16 and move it and the features to "cuda".

Training

Data

v1 was fine-tuned from the base model on 36.5 hours (27,966 utterances):

Before training, 21,615 duplicate utterances shared between these aggregated corpora were removed, along with 236 utterances whose text appears in a dev or test set. The dev set is the FLEURS km_kh and en_us dev splits (717 utterances).

v2 (this model) continues from v1 for one more pass over a larger mix:

• the v1 data, with FLEURS Khmer and khm-asr-open repeated three times;
• up to 100 hours of additional Khmer speech;
• additional English speech, not repeated;
• Khmer–English code-switched utterances.

As in v1, rows whose text appears in any dev or test set (including the code-switch test set) were removed, as were exact repeats.

Procedure

Targets are <|lang|><|transcribe|><|notimestamps|> text <|endoftext|>, so the model keeps its language identification but is not trained to predict timestamps.

To make the model hold up on real microphones, the training script corrupts 70 % of clips on the fly with random combinations of: DEMAND background noise (0–25 dB SNR), other speakers in the background, simulated room reverb, speed changes (0.9–1.1×), microphone hiss, a telephone band filter, and volume changes. SpecAugment is applied on top.

Limitations

• Timestamps: the model was trained without timestamp tokens. Take timing from a voice detector, as in the long-recording example above.
• Long audio: feed at most ~15 s of Khmer at a time; longer pieces can overflow the 448-token decoder and get cut off.
• Silence and noise: like other Whisper models, it can produce text for non-speech audio. Use a voice detector to skip silence.
• Coverage: much of the training audio is read speech (FLEURS, for example). Expect higher error rates on spontaneous conversation, strong regional accents, children's speech and far-field recordings.
• Quantisation: an int8 CTranslate2 export runs much faster on CPU but was clearly less accurate on Khmer in a spot check. Use the float model when accuracy matters.
• Use: do not use it for surveillance or to transcribe people without their consent. Review transcripts before relying on them for high-stakes decisions.

License and acknowledgements

The weights are released under the MIT license. The base model and OpenAI Whisper are released under Apache-2.0; check each training dataset's license before using this model commercially.

Thanks to OpenAI for Whisper, to PhanithLIM for the Khmer Whisper models this work starts from, and to the authors of the speech datasets listed above.

Citation

@misc{tha_asr_2026,
  title        = {Tha-ASR: Khmer and English speech recognition},
  author       = {Thareah},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Thareah/Tha-ASR}}
}