whisper-small-danish
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
βββββ βββββ
βββββ βββ ββββββ βββββββββββββββββββββ βββββββββββββββ βββββ
βββββ βββ ββββββ ββββββββββββββββββββββββββββββββββββββ βββββ
βββββ βββ ββ ββββββββββββββββββββββββββββββββββββ ββββββββ βββββ
βββββ ββββββββββββββββββββββββββββββββββββ ββββββ ββββββββ βββββ
βββββ βββββββββββββ βββββββββββββββββ βββββββββββ βββ βββββ
βββββ ββββββββ βββ βββββββββββββββββ βββββββββββ βββ βββββ
βββββ βββββ
βββββ ββββββ D A N I S H S P E E C H ββββββ βββββ
βββββ βββββ
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€ββββ
βββββ R E L E A S E I N F O [ MODEL / WEIGHTS ] βββββ
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€ββββ
βββββ βββββ
βββββ EDITION .......... SMALL βββββ
βββββ LANGUAGE ......... DANISH βββββ
βββββ STATUS ........... SOTA-ish βββββ
βββββ LAYERS ........... 12 ENCODER / 12 DECODER βββββ
βββββ MODEL WIDTH ...... 768 βββββ
βββββ AUDIO INPUT ...... 16 kHz MONO / 80 MEL BINS βββββ
βββββ TRAIN WINDOW ..... 30 SECONDS βββββ
βββββ VOCABULARY ....... 51,865 TOKENS βββββ
βββββ OPTIMIZER ........ MUON + ADAMW βββββ
βββββ TRAINING ......... 200,000 OPTIMIZER STEPS βββββ
βββββ TRAINED ON ....... NVIDIA GEFORCE RTX 4090 βββββ
βββββ βββββ
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€ββββ
βββββ 01 USAGE / 02 TRAINING / 03 EVALUATION βββββ
βββββ 04 LIMITATIONS / 05 LICENSE / 06 CREDITS βββββ
βββββ βββββ
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
A Danish fine-tune of Whisper Small. The architecture and tokenizer are unchanged. The released weights are the exponential moving average after 200,000 optimizer steps.
01 / USAGE
import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
model_id = "thorhojhus/whisper-small-danish"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if torch.cuda.is_available() else torch.float32
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
model_id, dtype=dtype, low_cpu_mem_usage=True,
).to(device)
transcribe = pipeline(
"automatic-speech-recognition",
model=model,
tokenizer=processor.tokenizer,
feature_extractor=processor.feature_extractor,
dtype=dtype,
device=device,
)
result = transcribe(
"audio.wav",
generate_kwargs={"language": "da", "task": "transcribe", "num_beams": 1},
)
print(result["text"])
02 / TRAINING
Training was started from stock Whisper Small. Filtered pseudo-labelled Danish speech was combined with the Danish portions of CoRal v3, Common Voice 26.0, FLEURS, FTSpeech and NST. Pseudo-labels were generated with whisper-large-v3-turbo-danish. The retained pool contained approximately 13,411 unique audio hours. Labels were filtered by confidence, no-speech probability, hallucination rules and a Danish/English vocabulary check.
Pseudo-labelled speech accounted for 100% of the sampling weight through step 50,000, 80% through step 140,000 and 70% thereafter. Human-labelled sources made up the remainder. Targets retained their casing, punctuation and numeral style; normalization was applied only for scoring.
The use of weak supervision and no-speech examples was inspired by the original Whisper paper. Waveform augmentation, SpecAugment and synthetic no-speech examples were included.
| Setting | Value |
|---|---|
| Steps | 200,000 |
| Optimizer | Muon for hidden matrices; AdamW for remaining parameters |
| Peak learning rate | Muon 5e-5; AdamW 2.5e-5 |
| Weight decay | 0.01 |
| Schedule | 1,000-step warmup, stable to 50,000, linear decay to 10% at 200,000 |
| EMA | Per-step decay 0.9999, applied every 50 steps |
| Precision | BF16 parameters with compensated updates; FP32 EMA |
| Maximum training segment | 30 seconds |
| Training hardware | One NVIDIA GeForce RTX 4090 |
| Logged active training time | Approximately 55.7 hours, excluding validation, checkpoint I/O and pauses |
The following figures are cumulative audio sampled on the final training trajectory, including repeated passes. They are not unique dataset durations.
| Source | Hours seen |
|---|---|
| CoRal conversation | 1,151.29 |
| CoRal read-aloud | 2,584.61 |
| Common Voice Danish | 139.94 |
| FLEURS Danish | 140.21 |
| FTSpeech | 4,182.40 |
| NST Danish | 1,938.15 |
| Pseudo-labelled data | 44,344.90 |
| Total | 54,481.50 |
Additional settings and provenance are recorded in training_recipe.json.
03 / EVALUATION
The checkpoint was evaluated using the public Danish ASR Leaderboard protocol with Transformers FP16 greedy Danish transcription. Standard decoding was used for short clips; timestamped sequential decoding was used above 30 seconds. Scoring included the leaderboard's Danish numeral normalization and did not remove filler words.
| Test set | Samples | WER (%) | CER (%) |
|---|---|---|---|
| CoRal conversation | 8,438 | 24.01 | 15.02 |
| CoRal read-aloud | 9,122 | 14.16 | 5.73 |
Common Voice (cv17_da) |
2,756 | 9.53 | 3.65 |
FLEURS da_dk |
930 | 9.94 | 3.94 |
FTSpeech test_balanced |
5,534 | 9.22 | 5.29 |
| Unweighted domain mean | 26,780 | 13.37 | 6.73 |
04 / LIMITATIONS
- Performance outside Danish has not been validated after fine-tuning.
- Long-form behavior was not directly trained and should be checked for the intended use.
- Silence, noise and unfamiliar names can still produce incorrect text.
05 / LICENSE
The model weights are released under the MIT License. Training data are not
redistributed and remain subject to their source terms. Attribution and the
upstream Whisper notice are included in THIRD_PARTY_NOTICES.md.
06 / CREDITS
@article{radford2022robust,
title={Robust Speech Recognition via Large-Scale Weak Supervision},
author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
journal={arXiv preprint arXiv:2212.04356},
year={2022}
}