whisper small finetuned Specaug whole data, speed non-native
This model is a fine-tuned version of openai/whisper-small on the LTL2021 dataset. It achieves the following results on the evaluation set:
- Loss: 0.4079
- Wer: 20.1022
- Cer: 14.2547
- Sub Rate: 4.8584
- Del Rate: 4.8082
- Ins Rate: 10.4356
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 64
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2048
- training_steps: 4000
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer | Cer | Sub Rate | Del Rate | Ins Rate |
|---|---|---|---|---|---|---|---|---|
| 5.1251 | 1.6087 | 500 | 2.9965 | 17.8718 | 12.9923 | 5.6252 | 6.6825 | 5.5641 |
| 3.1950 | 3.2158 | 1000 | 1.9016 | 17.2164 | 12.4481 | 5.0594 | 5.9027 | 6.2544 |
| 1.7948 | 4.8245 | 1500 | 1.0489 | 19.2459 | 14.1208 | 4.9567 | 5.8044 | 8.4848 |
| 0.9167 | 6.4316 | 2000 | 0.4769 | 18.9029 | 13.7232 | 4.9065 | 5.0682 | 8.9283 |
| 0.7342 | 8.0386 | 2500 | 0.4290 | 20.4539 | 14.8371 | 4.9393 | 4.7208 | 10.7939 |
| 0.7254 | 9.6473 | 3000 | 0.4157 | 19.1520 | 13.6238 | 4.8038 | 4.7689 | 9.5793 |
| 0.7044 | 11.2544 | 3500 | 0.4095 | 20.0891 | 14.2597 | 4.8344 | 4.8213 | 10.4334 |
| 0.6828 | 12.8631 | 4000 | 0.4079 | 20.1022 | 14.2547 | 4.8584 | 4.8082 | 10.4356 |
Framework versions
- Transformers 5.8.1
- Pytorch 2.5.1+cu121
- Datasets 4.8.5
- Tokenizers 0.22.2