Whisper Small Indonesian — Common Voice 27, 6-Hour Experiment

Summary

A fine-tune of openai/whisper-small on the Indonesian subset of Common Voice 27 (5,381 training clips and 400 evaluation clips, approximately 6 hours of audio), trained on a single RTX 2050 4 GB GPU.

Re-measured with a corrected decoding configuration, this checkpoint reaches WER 17.98% / wer_norm 14.61% on a random 30-clip sample of CV27-id, with zero decode failures. The training-time figure of 59.82% WER was a measurement artifact: the evaluation decodes ran with free language detection (see Measurement note).

Training details

The final checkpoint (step 1,011) shows higher training loss than step 337 and is not published. The complete per-step loss and metric history is included in trainer_state.json.

Evaluation

Metrics computed with jiwer. wer_norm is WER after lowercasing and punctuation removal.

Measurement note

During the original training run, generate() read generation_config.json, whose forced_decoder_ids left the language token unforced ([[1, null], [2, 50359]]), so evaluation decodes were free to pick any language. Clips where Whisper detected the wrong language produced looping non-Latin garbage, inflating WER far above the model's true quality. The published generation_config.json has since been corrected to force <|id|> at position 1; with that fix, unforced generation and explicit language forcing give identical results (17.98% on the sample above).

Caveats: the comparison against the baseline is indicative rather than apples-to-apples — different corpus versions (CV27 vs CV17) and sample sizes (30 clips vs 500 steps of evaluation). A full evaluation on the complete 400-clip split is pending; the exact training-time failure mechanism (fp16 decode, generation_max_length=225) is documented in the project repository.

Usage

from transformers import pipeline

pipe = pipeline(
    "automatic-speech-recognition",
    model="p-quincy/whisper-small-id-6h",
    chunk_length_s=30,
)
print(pipe("rapat.wav")["text"])

The repository's generation_config.json forces Indonesian decoding, so no language argument is needed. To be explicit, you can still pass language="Indonesian", task="transcribe" to generate().

Command line (from the accompanying project repository):

python src/transcription.py --audio rapat.wav --model p-quincy/whisper-small-id-6h

Limitations

• Evaluation is currently based on a 30-clip sample; full-split metrics are pending.
• Trained on approximately 6 hours of validated speech, so generalization to spontaneous speech, accents, and noisy meeting recordings is unverified.
• Indonesian only, single utterances of at most 30 seconds, no speaker diarization.
• The 59.82% WER recorded in the training logs does not reflect model quality; use the re-measured figures above.

Provenance

Training was run manually by the repository owner (the project's AGENTS.md prohibits automated training runs). The full training history is shipped in trainer_state.json so the numbers can be independently verified.