Shakespeare-T5-small
This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.8180
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 15
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 2.5071 | 0.0978 | 100 | 2.3866 |
| 1.924 | 0.1957 | 200 | 1.7508 |
| 1.3181 | 0.2935 | 300 | 1.4820 |
| 1.4516 | 0.3914 | 400 | 1.3604 |
| 1.3453 | 0.4892 | 500 | 1.2847 |
| 1.2216 | 0.5871 | 600 | 1.2377 |
| 1.5336 | 0.6849 | 700 | 1.2089 |
| 1.1803 | 0.7828 | 800 | 1.1853 |
| 1.352 | 0.8806 | 900 | 1.1659 |
| 1.25 | 0.9785 | 1000 | 1.1453 |
| 1.1877 | 1.0763 | 1100 | 1.1323 |
| 1.1111 | 1.1742 | 1200 | 1.1171 |
| 1.2295 | 1.2720 | 1300 | 1.1074 |
| 1.1602 | 1.3699 | 1400 | 1.0963 |
| 1.0951 | 1.4677 | 1500 | 1.0852 |
| 1.1102 | 1.5656 | 1600 | 1.0757 |
| 0.9648 | 1.6634 | 1700 | 1.0671 |
| 0.9534 | 1.7613 | 1800 | 1.0591 |
| 1.1413 | 1.8591 | 1900 | 1.0526 |
| 1.1051 | 1.9569 | 2000 | 1.0470 |
| 1.1152 | 2.0548 | 2100 | 1.0410 |
| 1.233 | 2.1526 | 2200 | 1.0367 |
| 0.9705 | 2.2505 | 2300 | 1.0316 |
| 0.8814 | 2.3483 | 2400 | 1.0250 |
| 0.967 | 2.4462 | 2500 | 1.0208 |
| 1.0735 | 2.5440 | 2600 | 1.0168 |
| 1.1725 | 2.6419 | 2700 | 1.0098 |
| 1.0661 | 2.7397 | 2800 | 1.0073 |
| 1.0255 | 2.8376 | 2900 | 1.0022 |
| 1.0509 | 2.9354 | 3000 | 0.9992 |
| 1.087 | 3.0333 | 3100 | 0.9942 |
| 1.0105 | 3.1311 | 3200 | 0.9897 |
| 1.0746 | 3.2290 | 3300 | 0.9860 |
| 1.0384 | 3.3268 | 3400 | 0.9833 |
| 1.0473 | 3.4247 | 3500 | 0.9791 |
| 1.0041 | 3.5225 | 3600 | 0.9744 |
| 1.0455 | 3.6204 | 3700 | 0.9713 |
| 1.0431 | 3.7182 | 3800 | 0.9675 |
| 0.8924 | 3.8160 | 3900 | 0.9645 |
| 0.8299 | 3.9139 | 4000 | 0.9611 |
| 0.8729 | 4.0117 | 4100 | 0.9583 |
| 0.8513 | 4.1096 | 4200 | 0.9556 |
| 1.0433 | 4.2074 | 4300 | 0.9512 |
| 0.9206 | 4.3053 | 4400 | 0.9491 |
| 0.9408 | 4.4031 | 4500 | 0.9461 |
| 0.9793 | 4.5010 | 4600 | 0.9433 |
| 0.8709 | 4.5988 | 4700 | 0.9402 |
| 0.9612 | 4.6967 | 4800 | 0.9370 |
| 1.0882 | 4.7945 | 4900 | 0.9346 |
| 0.9773 | 4.8924 | 5000 | 0.9331 |
| 0.9365 | 4.9902 | 5100 | 0.9302 |
| 0.7646 | 5.0881 | 5200 | 0.9276 |
| 1.0583 | 5.1859 | 5300 | 0.9246 |
| 0.9999 | 5.2838 | 5400 | 0.9223 |
| 0.9926 | 5.3816 | 5500 | 0.9207 |
| 0.8495 | 5.4795 | 5600 | 0.9194 |
| 1.0125 | 5.5773 | 5700 | 0.9163 |
| 0.858 | 5.6751 | 5800 | 0.9145 |
| 0.9271 | 5.7730 | 5900 | 0.9121 |
| 0.9491 | 5.8708 | 6000 | 0.9111 |
| 0.998 | 5.9687 | 6100 | 0.9079 |
| 1.0376 | 6.0665 | 6200 | 0.9048 |
| 0.9302 | 6.1644 | 6300 | 0.9037 |
| 0.8533 | 6.2622 | 6400 | 0.9017 |
| 0.9285 | 6.3601 | 6500 | 0.8983 |
| 0.8977 | 6.4579 | 6600 | 0.8961 |
| 0.9745 | 6.5558 | 6700 | 0.8953 |
| 0.9405 | 6.6536 | 6800 | 0.8925 |
| 0.9929 | 6.7515 | 6900 | 0.8917 |
| 0.7314 | 6.8493 | 7000 | 0.8897 |
| 0.8764 | 6.9472 | 7100 | 0.8875 |
| 0.8443 | 7.0450 | 7200 | 0.8863 |
| 0.7612 | 7.1429 | 7300 | 0.8847 |
| 1.0069 | 7.2407 | 7400 | 0.8829 |
| 1.0436 | 7.3386 | 7500 | 0.8816 |
| 0.8622 | 7.4364 | 7600 | 0.8812 |
| 0.8886 | 7.5342 | 7700 | 0.8783 |
| 0.9365 | 7.6321 | 7800 | 0.8763 |
| 0.8307 | 7.7299 | 7900 | 0.8749 |
| 0.7874 | 7.8278 | 8000 | 0.8731 |
| 0.8606 | 7.9256 | 8100 | 0.8711 |
| 0.9094 | 8.0235 | 8200 | 0.8698 |
| 0.8108 | 8.1213 | 8300 | 0.8683 |
| 1.1008 | 8.2192 | 8400 | 0.8665 |
| 0.8168 | 8.3170 | 8500 | 0.8642 |
| 0.9125 | 8.4149 | 8600 | 0.8635 |
| 0.9218 | 8.5127 | 8700 | 0.8615 |
| 0.9688 | 8.6106 | 8800 | 0.8596 |
| 0.8679 | 8.7084 | 8900 | 0.8582 |
| 0.7928 | 8.8063 | 9000 | 0.8574 |
| 0.7947 | 8.9041 | 9100 | 0.8561 |
| 0.9055 | 9.0020 | 9200 | 0.8554 |
| 1.0261 | 9.0998 | 9300 | 0.8538 |
| 0.8588 | 9.1977 | 9400 | 0.8530 |
| 0.8789 | 9.2955 | 9500 | 0.8526 |
| 0.9111 | 9.3933 | 9600 | 0.8511 |
| 0.7969 | 9.4912 | 9700 | 0.8503 |
| 0.8683 | 9.5890 | 9800 | 0.8492 |
| 0.8884 | 9.6869 | 9900 | 0.8479 |
| 0.9843 | 9.7847 | 10000 | 0.8464 |
| 0.7575 | 9.8826 | 10100 | 0.8449 |
| 0.887 | 9.9804 | 10200 | 0.8441 |
| 0.9358 | 10.0783 | 10300 | 0.8428 |
| 0.7544 | 10.1761 | 10400 | 0.8422 |
| 0.7679 | 10.2740 | 10500 | 0.8411 |
| 0.7898 | 10.3718 | 10600 | 0.8406 |
| 0.8641 | 10.4697 | 10700 | 0.8397 |
| 0.9489 | 10.5675 | 10800 | 0.8385 |
| 0.8384 | 10.6654 | 10900 | 0.8380 |
| 0.8114 | 10.7632 | 11000 | 0.8372 |
| 0.8874 | 10.8611 | 11100 | 0.8361 |
| 0.7843 | 10.9589 | 11200 | 0.8349 |
| 0.7065 | 11.0568 | 11300 | 0.8343 |
| 0.803 | 11.1546 | 11400 | 0.8339 |
| 0.827 | 11.2524 | 11500 | 0.8328 |
| 0.8149 | 11.3503 | 11600 | 0.8319 |
| 0.8248 | 11.4481 | 11700 | 0.8314 |
| 0.7901 | 11.5460 | 11800 | 0.8308 |
| 0.7693 | 11.6438 | 11900 | 0.8301 |
| 0.8768 | 11.7417 | 12000 | 0.8293 |
| 0.7997 | 11.8395 | 12100 | 0.8292 |
| 0.8565 | 11.9374 | 12200 | 0.8282 |
| 0.7635 | 12.0352 | 12300 | 0.8274 |
| 0.7206 | 12.1331 | 12400 | 0.8265 |
| 0.8557 | 12.2309 | 12500 | 0.8263 |
| 0.7447 | 12.3288 | 12600 | 0.8256 |
| 0.9202 | 12.4266 | 12700 | 0.8248 |
| 0.9656 | 12.5245 | 12800 | 0.8242 |
| 0.8597 | 12.6223 | 12900 | 0.8236 |
| 0.7596 | 12.7202 | 13000 | 0.8231 |
| 0.7913 | 12.8180 | 13100 | 0.8226 |
| 0.7493 | 12.9159 | 13200 | 0.8223 |
| 0.6831 | 13.0137 | 13300 | 0.8219 |
| 0.7939 | 13.1115 | 13400 | 0.8217 |
| 0.7227 | 13.2094 | 13500 | 0.8215 |
| 0.8662 | 13.3072 | 13600 | 0.8212 |
| 0.686 | 13.4051 | 13700 | 0.8210 |
| 0.7837 | 13.5029 | 13800 | 0.8206 |
| 0.8727 | 13.6008 | 13900 | 0.8202 |
| 0.7521 | 13.6986 | 14000 | 0.8199 |
| 0.7101 | 13.7965 | 14100 | 0.8196 |
| 0.9385 | 13.8943 | 14200 | 0.8193 |
| 0.7965 | 13.9922 | 14300 | 0.8191 |
| 0.8829 | 14.0900 | 14400 | 0.8188 |
| 0.8408 | 14.1879 | 14500 | 0.8186 |
| 0.8808 | 14.2857 | 14600 | 0.8185 |
| 0.8918 | 14.3836 | 14700 | 0.8184 |
| 0.756 | 14.4814 | 14800 | 0.8182 |
| 0.8249 | 14.5793 | 14900 | 0.8181 |
| 0.721 | 14.6771 | 15000 | 0.8181 |
| 0.8631 | 14.7750 | 15100 | 0.8180 |
| 0.789 | 14.8728 | 15200 | 0.8180 |
| 0.6724 | 14.9706 | 15300 | 0.8180 |
Framework versions
- Transformers 4.52.3
- Pytorch 2.6.0+cu124
- Datasets 2.14.4
- Tokenizers 0.21.1