openai/whisper-small
This model is a fine-tuned version of openai/whisper-small on the pphuc25/VietMed-split-8-2 dataset. It achieves the following results on the evaluation set:
- Loss: 0.8271
- Wer: 58.2317
- Cer: 46.8054
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 20
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer | Cer |
|---|---|---|---|---|---|
| 0.5609 | 1.0 | 569 | 0.5509 | 95.7642 | 78.5963 |
| 0.4081 | 2.0 | 1138 | 0.5360 | 68.6290 | 56.1987 |
| 0.278 | 3.0 | 1707 | 0.5530 | 34.9844 | 29.3423 |
| 0.187 | 4.0 | 2276 | 0.5937 | 71.8946 | 56.4435 |
| 0.1106 | 5.0 | 2845 | 0.6325 | 41.3839 | 34.3220 |
| 0.0761 | 6.0 | 3414 | 0.6773 | 44.9131 | 38.3312 |
| 0.0398 | 7.0 | 3983 | 0.6962 | 95.0394 | 82.5127 |
| 0.0317 | 8.0 | 4552 | 0.7284 | 55.6288 | 46.0103 |
| 0.0187 | 9.0 | 5121 | 0.7376 | 55.4347 | 44.4601 |
| 0.0099 | 10.0 | 5690 | 0.7598 | 53.4066 | 42.1512 |
| 0.01 | 11.0 | 6259 | 0.7753 | 82.2222 | 67.2734 |
| 0.0032 | 12.0 | 6828 | 0.7852 | 65.0119 | 47.9815 |
| 0.0019 | 13.0 | 7397 | 0.7838 | 53.5530 | 40.5203 |
| 0.0037 | 14.0 | 7966 | 0.7975 | 50.7377 | 40.2000 |
| 0.0007 | 15.0 | 8535 | 0.8026 | 45.6233 | 36.3097 |
| 0.0013 | 16.0 | 9104 | 0.8134 | 56.5001 | 44.4010 |
| 0.0002 | 17.0 | 9673 | 0.8082 | 57.5654 | 46.8896 |
| 0.0002 | 18.0 | 10242 | 0.8203 | 62.1234 | 49.8199 |
| 0.0001 | 19.0 | 10811 | 0.8270 | 57.0346 | 46.1639 |
| 0.0001 | 20.0 | 11380 | 0.8271 | 58.2317 | 46.8054 |
Framework versions
- Transformers 4.41.1
- Pytorch 2.3.0
- Datasets 2.19.1
- Tokenizers 0.19.1