Whistle fine-tuned for Korean
This is a fine-tune of Cactus-Compute/whistle that adds Korean to the original seven languages: English, German, French, Spanish, Italian, Dutch and Polish. It is still a single on-device file (whistle.cact, 30 MB, 4-bit) that runs on the same CPU engine as the original. The model detects all eight languages, including Korean, unless you set the language yourself.
Results
All scores come from the shipped 4-bit whistle.cact, run on the cactus-needle engine at its defaults. Lower is better.
- Korean: CER (character error rate) with spaces removed.
- Other languages: WER (word error rate) after the Whisper normalizer.
| Test set | Original Whistle | This model |
|---|---|---|
| Zeroth-Korean test, 457 clips, ko CER | — (no Korean) | 18.5% |
| FLEURS ko_kr test, 382 clips, ko CER | 264% (answers in other languages) | 32.2% |
| FLEURS en_us test, 100 clips, WER | 16.1% | 16.2% |
| FLEURS de_de test, 100 clips, WER | 21.9% | 22.2% |
| FLEURS fr_fr test, 100 clips, WER | 23.4% | 25.9% |
| FLEURS es_419 test, 100 clips, WER | 10.6% | 10.8% |
| FLEURS it_it test, 100 clips, WER | 13.9% | 13.9% |
| FLEURS nl_nl test, 100 clips, WER | 27.0% | 29.8% |
| FLEURS pl_pl test, 100 clips, WER | 35.5% | 35.8% |
Language detection was 99% correct on Korean FLEURS clips and 97–100% correct on the other seven languages.
Fine-tuning on Korean data alone pushed the other seven languages to 92–214% WER and made the model label all speech as Korean. Mixing in some data from the original languages, as described below, avoided that.
Usage
pip install cactus-needle
import needle
from huggingface_hub import hf_hub_download
path = hf_hub_download("Sky-Kim/whistle-finetuned-ko", "whistle.cact")
whistle = needle.Whistle(path)
print(whistle.transcribe("clip.wav")) # detects the language, Korean included
print(whistle.transcribe("clip.wav", language="ko")) # forces Korean
The input must be 16 kHz mono audio. The file also works with the needle runner and C API, the same way the original does:
needle --model whistle.cact --audio clip.wav
Training
- Method: LoRA (rank 32, alpha 64) on the attention projections, with the base weights frozen. Training used 4-bit weights and 8-bit activations, matching the exported file. The run used
needle whistle finetunefrom cactus-needle, plus a wrapper that saves the adapter after every epoch. - Vocabulary: the original tokenizer had no Hangul, so every syllable became three byte-fallback tokens. 1,202 Hangul syllables from the training transcripts were added (2,404 new pieces: each syllable with and without a leading space), and
kowas added as a ninth language tag. This cut the median token count per Korean sentence from 133 to 40. - Data: a pool of 29,263 clips, of which 28,678 were trained on.
- The full Zeroth-Korean train split: 22,263 clips, about 51 h.
- A random 1,000 clips per original language from FLEURS train, using
raw_transcriptionso casing and punctuation are kept. That is about a third of each language's train split, which holds 2,602–3,193 clips; the slice is there only to keep the original languages from degrading. - A random 2% of the pool (585 clips, Korean and other languages mixed) was held out for validation and never trained on.
- No test split was used for training: not Zeroth-Korean test, FLEURS ko_kr test, or any language's FLEURS test.
- Schedule: AdamW with a learning rate of 5e-4 (10× for the new vocabulary rows), batch size 16, warmup and cosine decay over 3 epochs. This release is the epoch 2 checkpoint, which scored slightly better on Korean FLEURS than epoch 3 (32.2% vs 33.5% CER), while the other languages scored about the same.
- Hardware: a single Apple-silicon CPU.
The LoRA adapter is in adapter/whistle_ko_lora.safetensors. To rebuild the .cact from it, for example for another platform folder or decoder depth, run the command below. It downloads the original checkpoint if it is missing, and the result is byte-identical to the shipped whistle.cact.
pip install "cactus-needle[train] @ git+https://github.com/cactus-compute/needle" # whistle build/finetune are newer than the PyPI release used here
needle whistle build --lora adapter/whistle_ko_lora.safetensors --out whistle.cact
Limitations
- Numbers: Zeroth writes numbers out in Hangul ("삼 월"), while FLEURS uses digits ("3월"), so the model often spells numbers in Hangul. This accounts for part of the FLEURS CER.
- Proper nouns: foreign names and loanwords are often misspelled. For example, "델 포트로" comes out as "테보트로".
- Training data size: the Korean data is about 51 h of read news-style speech. Accuracy on conversational speech, dialects or noisy audio has not been measured.
- Other languages: on the 100-clip FLEURS samples, French and Dutch are 2–3 WER points worse than the original. The other languages are within 0.3 points.
License and attribution
- Model: a derivative of Cactus-Compute/whistle by Cactus Compute, Inc., released under the same Apache License 2.0 (
LICENSE). - Training data: Zeroth-Korean, CC BY 4.0, and FLEURS, CC BY 4.0.
If you use Whistle, please cite the original work:
@misc{whistle_2026,
title = {Whistle: Speech Recognition for Tiny Devices},
author = {Mroz, Jakub and Ndubuaku, Henry and Mosoyan, Karen and Cylich, Noah and
Kumar, Satyajit and Sandhu, Parkirat and Shemet, Roman and Lee, Justin H.},
year = {2026},
organization = {Cactus Compute, Inc.},
howpublished = {\url{https://github.com/cactus-compute/needle}}
}