whisper-small-CV-Fleurs-lg-1hrs-v1
This model is a fine-tuned version of openai/whisper-small on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.5981
- Wer: 0.8787
- Cer: 0.3264
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 4
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Use OptimizerNames.ADAMW_HF with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer | Cer |
|---|---|---|---|---|---|
| 5.6051 | 1.0 | 66 | 3.5145 | 1.4569 | 0.6891 |
| 2.4999 | 2.0 | 132 | 2.0266 | 1.1776 | 0.4738 |
| 1.5987 | 3.0 | 198 | 1.6438 | 1.0026 | 0.3884 |
| 1.1473 | 4.0 | 264 | 1.4459 | 0.9933 | 0.3876 |
| 0.7941 | 5.0 | 330 | 1.3575 | 1.0070 | 0.3922 |
| 0.5137 | 6.0 | 396 | 1.3371 | 1.0377 | 0.4016 |
| 0.2992 | 7.0 | 462 | 1.3442 | 1.0814 | 0.4432 |
| 0.1564 | 8.0 | 528 | 1.3580 | 1.1707 | 0.4875 |
| 0.0918 | 9.0 | 594 | 1.3761 | 1.0757 | 0.4521 |
| 0.0594 | 10.0 | 660 | 1.3911 | 1.1061 | 0.4463 |
| 0.0421 | 11.0 | 726 | 1.4346 | 1.0884 | 0.4501 |
| 0.0307 | 12.0 | 792 | 1.4065 | 1.1398 | 0.4760 |
| 0.0244 | 13.0 | 858 | 1.4255 | 0.9986 | 0.3835 |
| 0.0166 | 14.0 | 924 | 1.4470 | 0.9827 | 0.4068 |
| 0.0113 | 15.0 | 990 | 1.4665 | 1.0230 | 0.4303 |
| 0.0129 | 16.0 | 1056 | 1.4771 | 1.0429 | 0.4329 |
| 0.0081 | 17.0 | 1122 | 1.4891 | 0.9374 | 0.3507 |
| 0.0078 | 18.0 | 1188 | 1.4732 | 0.9948 | 0.3844 |
| 0.0043 | 19.0 | 1254 | 1.4902 | 0.9031 | 0.3429 |
| 0.0015 | 20.0 | 1320 | 1.4938 | 0.8725 | 0.3073 |
| 0.0014 | 21.0 | 1386 | 1.5089 | 0.8962 | 0.3254 |
| 0.0006 | 22.0 | 1452 | 1.5250 | 0.8717 | 0.3070 |
| 0.0005 | 23.0 | 1518 | 1.5295 | 0.8920 | 0.3174 |
| 0.0004 | 24.0 | 1584 | 1.5374 | 0.9067 | 0.3286 |
| 0.0004 | 25.0 | 1650 | 1.5447 | 0.9138 | 0.3367 |
| 0.0004 | 26.0 | 1716 | 1.5517 | 0.9078 | 0.3349 |
| 0.0003 | 27.0 | 1782 | 1.5572 | 0.8969 | 0.3329 |
| 0.0003 | 28.0 | 1848 | 1.5627 | 0.9181 | 0.3504 |
| 0.0003 | 29.0 | 1914 | 1.5684 | 0.9089 | 0.3423 |
| 0.0003 | 30.0 | 1980 | 1.5735 | 0.9190 | 0.3513 |
| 0.0003 | 31.0 | 2046 | 1.5790 | 0.8971 | 0.3363 |
| 0.0002 | 32.0 | 2112 | 1.5846 | 0.8979 | 0.3365 |
| 0.0002 | 33.0 | 2178 | 1.5893 | 0.8793 | 0.3235 |
| 0.0002 | 34.0 | 2244 | 1.5948 | 0.8864 | 0.3316 |
| 0.0002 | 35.0 | 2310 | 1.5981 | 0.8787 | 0.3264 |
Framework versions
- Transformers 4.48.1
- Pytorch 2.5.1+cu124
- Datasets 3.2.0
- Tokenizers 0.21.0