120min_wav2vec_xls-r53_FT
This model is a fine-tuned version of jonatasgrosman/wav2vec2-large-xlsr-53-arabic on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.1684
- Wer: 0.6547
- Cer: 0.2177
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 12
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer | Cer |
|---|---|---|---|---|---|
| 1.8782 | 0.1908 | 100 | 1.3708 | 0.9188 | 0.3620 |
| 1.3601 | 0.3817 | 200 | 1.1605 | 0.8503 | 0.3078 |
| 1.3002 | 0.5725 | 300 | 1.0615 | 0.8377 | 0.2959 |
| 1.2639 | 0.7634 | 400 | 1.0643 | 0.8093 | 0.2844 |
| 1.1603 | 0.9542 | 500 | 1.0018 | 0.7723 | 0.2685 |
| 1.1299 | 1.1450 | 600 | 1.0108 | 0.7599 | 0.2646 |
| 0.9707 | 1.3359 | 700 | 0.9639 | 0.7449 | 0.2558 |
| 1.0436 | 1.5267 | 800 | 0.9838 | 0.7850 | 0.2675 |
| 0.9747 | 1.7176 | 900 | 1.0015 | 0.7400 | 0.2527 |
| 0.9765 | 1.9084 | 1000 | 0.9177 | 0.7264 | 0.2475 |
| 0.9291 | 2.0992 | 1100 | 0.9657 | 0.7285 | 0.2469 |
| 0.8804 | 2.2901 | 1200 | 0.9735 | 0.7219 | 0.2446 |
| 0.7731 | 2.4809 | 1300 | 0.9996 | 0.7102 | 0.2401 |
| 0.8597 | 2.6718 | 1400 | 1.0208 | 0.7053 | 0.2398 |
| 0.8478 | 2.8626 | 1500 | 0.9148 | 0.6933 | 0.2322 |
| 0.7741 | 3.0534 | 1600 | 1.0133 | 0.6992 | 0.2352 |
| 0.6887 | 3.2443 | 1700 | 0.9603 | 0.6912 | 0.2309 |
| 0.712 | 3.4351 | 1800 | 1.0089 | 0.6882 | 0.2311 |
| 0.7073 | 3.6260 | 1900 | 0.9700 | 0.6855 | 0.2323 |
| 0.7667 | 3.8168 | 2000 | 1.0454 | 0.6954 | 0.2311 |
| 0.7134 | 4.0076 | 2100 | 0.9957 | 0.6866 | 0.2257 |
| 0.5991 | 4.1985 | 2200 | 0.9846 | 0.6862 | 0.2266 |
| 0.622 | 4.3893 | 2300 | 1.0161 | 0.6748 | 0.2256 |
| 0.6062 | 4.5802 | 2400 | 1.0280 | 0.6881 | 0.2292 |
| 0.6094 | 4.7710 | 2500 | 0.9586 | 0.6694 | 0.2240 |
| 0.6701 | 4.9618 | 2600 | 0.9728 | 0.6706 | 0.2239 |
| 0.586 | 5.1527 | 2700 | 1.0369 | 0.6667 | 0.2241 |
| 0.6001 | 5.3435 | 2800 | 1.1335 | 0.6670 | 0.2243 |
| 0.4735 | 5.5344 | 2900 | 1.0438 | 0.6733 | 0.2233 |
| 0.5801 | 5.7252 | 3000 | 1.0320 | 0.6722 | 0.2234 |
| 0.5684 | 5.9160 | 3100 | 0.9592 | 0.6664 | 0.2222 |
| 0.4922 | 6.1069 | 3200 | 1.0551 | 0.6647 | 0.2203 |
| 0.4317 | 6.2977 | 3300 | 1.1491 | 0.6581 | 0.2200 |
| 0.5079 | 6.4885 | 3400 | 1.0728 | 0.6565 | 0.2203 |
| 0.4498 | 6.6794 | 3500 | 1.0398 | 0.6579 | 0.2178 |
| 0.5414 | 6.8702 | 3600 | 1.0863 | 0.6616 | 0.2195 |
| 0.4707 | 7.0611 | 3700 | 1.1589 | 0.6540 | 0.2190 |
| 0.4554 | 7.2519 | 3800 | 1.1677 | 0.6591 | 0.2201 |
| 0.4644 | 7.4427 | 3900 | 1.0799 | 0.6555 | 0.2181 |
| 0.4021 | 7.6336 | 4000 | 1.1018 | 0.6570 | 0.2185 |
| 0.458 | 7.8244 | 4100 | 1.1470 | 0.6475 | 0.2154 |
| 0.4341 | 8.0153 | 4200 | 1.1655 | 0.6519 | 0.2183 |
| 0.3751 | 8.2061 | 4300 | 1.2057 | 0.6568 | 0.2190 |
| 0.3804 | 8.3969 | 4400 | 1.1528 | 0.6473 | 0.2157 |
| 0.3895 | 8.5878 | 4500 | 1.1330 | 0.6493 | 0.2184 |
| 0.4174 | 8.7786 | 4600 | 1.1791 | 0.6479 | 0.2171 |
| 0.4163 | 8.9695 | 4700 | 1.2067 | 0.6620 | 0.2172 |
| 0.3946 | 9.1603 | 4800 | 1.1451 | 0.6549 | 0.2201 |
| 0.36 | 9.3511 | 4900 | 1.1684 | 0.6547 | 0.2177 |
Framework versions
- Transformers 4.41.1
- Pytorch 2.9.0+cu126
- Datasets 4.0.0
- Tokenizers 0.19.1