whisper-small-basque
This model is a fine-tuned version of openai/whisper-small on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.2714
- Wer: 18.3719
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 128
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- training_steps: 10000
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer |
|---|---|---|---|---|
| 0.4242 | 0.11 | 500 | 0.5947 | 47.5547 |
| 0.2999 | 0.21 | 1000 | 0.4454 | 34.5728 |
| 0.2505 | 0.32 | 1500 | 0.3934 | 28.3430 |
| 0.22 | 0.42 | 2000 | 0.3644 | 23.5473 |
| 0.2131 | 0.53 | 2500 | 0.3424 | 22.3772 |
| 0.2006 | 0.63 | 3000 | 0.3280 | 22.1647 |
| 0.1779 | 0.74 | 3500 | 0.3161 | 21.0050 |
| 0.182 | 0.84 | 4000 | 0.3064 | 20.5056 |
| 0.1724 | 0.95 | 4500 | 0.2966 | 20.1589 |
| 0.1319 | 1.05 | 5000 | 0.2923 | 19.8163 |
| 0.1415 | 1.16 | 5500 | 0.2891 | 19.7441 |
| 0.1362 | 1.26 | 6000 | 0.2865 | 19.6843 |
| 0.1327 | 1.37 | 6500 | 0.2828 | 18.9393 |
| 0.1336 | 1.47 | 7000 | 0.2795 | 19.5790 |
| 0.1326 | 1.58 | 7500 | 0.2760 | 18.4957 |
| 0.1361 | 1.68 | 8000 | 0.2753 | 18.8217 |
| 0.1275 | 1.79 | 8500 | 0.2735 | 18.5988 |
| 0.1288 | 1.89 | 9000 | 0.2714 | 18.3244 |
| 0.1185 | 2.0 | 9500 | 0.2705 | 18.7763 |
| 0.1023 | 2.1 | 10000 | 0.2714 | 18.3719 |
Framework versions
- Transformers 4.36.0
- Pytorch 2.1.1+cu121
- Datasets 2.8.0
- Tokenizers 0.15.2