trocr-bigram4-BY-350k
This model is a fine-tuned version of cyttic/exp2-frozen-benyehuda-cont on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.4875
- Cer: 0.0226
- Wer: 0.0665
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 6525
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss | Cer | Wer |
|---|---|---|---|---|---|
| 3.9106 | 0.0920 | 2000 | 1.7790 | 0.2020 | 0.4168 |
| 3.0313 | 0.1839 | 4000 | 1.4338 | 0.1336 | 0.3006 |
| 3.0083 | 0.2759 | 6000 | 1.3042 | 0.1090 | 0.2592 |
| 2.6042 | 0.3678 | 8000 | 1.1926 | 0.0968 | 0.2300 |
| 2.3764 | 0.4598 | 10000 | 1.0953 | 0.0801 | 0.2014 |
| 2.2069 | 0.5517 | 12000 | 1.0291 | 0.0725 | 0.1883 |
| 2.0087 | 0.6437 | 14000 | 0.9284 | 0.0617 | 0.1665 |
| 2.0671 | 0.7356 | 16000 | 0.8690 | 0.0591 | 0.1568 |
| 2.0193 | 0.8276 | 18000 | 0.8294 | 0.0487 | 0.1382 |
| 1.7531 | 0.9195 | 20000 | 0.7896 | 0.0467 | 0.1299 |
| 1.2986 | 1.0115 | 22000 | 0.7534 | 0.0414 | 0.1188 |
| 1.4100 | 1.1034 | 24000 | 0.7350 | 0.0415 | 0.1174 |
| 1.3172 | 1.1954 | 26000 | 0.7038 | 0.0380 | 0.1083 |
| 1.2106 | 1.2874 | 28000 | 0.6793 | 0.0350 | 0.1041 |
| 1.2120 | 1.3793 | 30000 | 0.6600 | 0.0334 | 0.0970 |
| 1.1722 | 1.4713 | 32000 | 0.6456 | 0.0316 | 0.0928 |
| 1.2070 | 1.5632 | 34000 | 0.6240 | 0.0305 | 0.0906 |
| 1.0933 | 1.6552 | 36000 | 0.6003 | 0.0288 | 0.0840 |
| 1.1695 | 1.7471 | 38000 | 0.5832 | 0.0276 | 0.0824 |
| 1.1884 | 1.8391 | 40000 | 0.5703 | 0.0282 | 0.0808 |
| 1.0362 | 1.9310 | 42000 | 0.5566 | 0.0251 | 0.0747 |
| 0.9332 | 2.0230 | 44000 | 0.5483 | 0.0275 | 0.0796 |
| 0.8579 | 2.1149 | 46000 | 0.5384 | 0.0265 | 0.0776 |
| 0.8650 | 2.2069 | 48000 | 0.5348 | 0.0255 | 0.0725 |
| 0.8576 | 2.2989 | 50000 | 0.5217 | 0.0239 | 0.0707 |
| 0.8781 | 2.3908 | 52000 | 0.5173 | 0.0257 | 0.0729 |
| 0.8028 | 2.4828 | 54000 | 0.5089 | 0.0224 | 0.0679 |
| 0.8699 | 2.5747 | 56000 | 0.5053 | 0.0240 | 0.0697 |
| 0.8827 | 2.6667 | 58000 | 0.4950 | 0.0229 | 0.0670 |
| 0.7242 | 2.7586 | 60000 | 0.4952 | 0.0228 | 0.0688 |
| 0.8109 | 2.8506 | 62000 | 0.4892 | 0.0231 | 0.0675 |
| 0.7782 | 2.9425 | 64000 | 0.4875 | 0.0223 | 0.0662 |
| 0.7634 | 3.0 | 65250 | 0.4875 | 0.0226 | 0.0665 |
Framework versions
- Transformers 5.15.0
- Pytorch 2.11.0+cu128
- Datasets 5.0.1
- Tokenizers 0.22.2