Whisper Small - Toki Pona - Synthetic Test 1

This experimental model is a fine-tuned version of openai/whisper-small on a mix of custom synthetic data and Common Voice 23.0 - Toki Pona. It achieves the following results on the evaluation set, made up exclusively of Common Voice data:

• Loss: 0.1691
• Wer: 10.0408

Model description

This is an experimental model trained for speech recognition for Toki Pona.

As the original model is multilingual with explicit language specification tokens, we have replaced the Czech (cs) language with Toki Pona, as we have determined it to have the closest phonetics.

The model's performance for other languages seems to have been at least partially preserved, but no testing has been done for other languages.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 1600
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.50.3
• Pytorch 2.9.0+cu126
• Datasets 3.6.0
• Tokenizers 0.21.4