mt5_small_lg_inf_en_v1
This model is a fine-tuned version of MubarakB/mt5_small_lg_en on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.4187
- Bleu: 0.2171
- Gen Len: 9.0204
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
Training results
| Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len |
|---|---|---|---|---|---|
| No log | 1.0 | 138 | 0.4669 | 0.0658 | 9.3837 |
| No log | 2.0 | 276 | 0.4559 | 0.132 | 8.0245 |
| No log | 3.0 | 414 | 0.4507 | 0.2112 | 8.1592 |
| 0.4726 | 4.0 | 552 | 0.4472 | 0.2144 | 8.0367 |
| 0.4726 | 5.0 | 690 | 0.4445 | 0.2134 | 8.0082 |
| 0.4726 | 6.0 | 828 | 0.4425 | 0.3274 | 7.8612 |
| 0.4726 | 7.0 | 966 | 0.4405 | 0.3378 | 7.5959 |
| 0.447 | 8.0 | 1104 | 0.4390 | 0.3304 | 7.3918 |
| 0.447 | 9.0 | 1242 | 0.4378 | 0.3285 | 7.3673 |
| 0.447 | 10.0 | 1380 | 0.4362 | 0.3147 | 7.6694 |
| 0.4398 | 11.0 | 1518 | 0.4350 | 0.3181 | 7.4163 |
| 0.4398 | 12.0 | 1656 | 0.4341 | 0.3166 | 7.5224 |
| 0.4398 | 13.0 | 1794 | 0.4330 | 0.3178 | 7.5592 |
| 0.4398 | 14.0 | 1932 | 0.4318 | 0.2157 | 7.8204 |
| 0.4313 | 15.0 | 2070 | 0.4312 | 0.3169 | 8.1388 |
| 0.4313 | 16.0 | 2208 | 0.4307 | 0.3169 | 7.9633 |
| 0.4313 | 17.0 | 2346 | 0.4297 | 0.3064 | 8.2245 |
| 0.4313 | 18.0 | 2484 | 0.4293 | 0.2045 | 8.2776 |
| 0.4262 | 19.0 | 2622 | 0.4286 | 0.3027 | 8.4367 |
| 0.4262 | 20.0 | 2760 | 0.4280 | 0.2042 | 8.5061 |
| 0.4262 | 21.0 | 2898 | 0.4274 | 0.3033 | 8.5633 |
| 0.4214 | 22.0 | 3036 | 0.4272 | 0.3019 | 8.7714 |
| 0.4214 | 23.0 | 3174 | 0.4264 | 0.3051 | 8.649 |
| 0.4214 | 24.0 | 3312 | 0.4263 | 0.3021 | 8.8367 |
| 0.4214 | 25.0 | 3450 | 0.4254 | 0.2981 | 8.8204 |
| 0.4161 | 26.0 | 3588 | 0.4251 | 0.2992 | 8.8776 |
| 0.4161 | 27.0 | 3726 | 0.4248 | 0.3044 | 8.8571 |
| 0.4161 | 28.0 | 3864 | 0.4246 | 0.3 | 8.8776 |
| 0.4124 | 29.0 | 4002 | 0.4246 | 0.2998 | 8.8163 |
| 0.4124 | 30.0 | 4140 | 0.4239 | 0.2983 | 9.0857 |
| 0.4124 | 31.0 | 4278 | 0.4234 | 0.2988 | 9.0163 |
| 0.4124 | 32.0 | 4416 | 0.4233 | 0.2996 | 8.8816 |
| 0.4087 | 33.0 | 4554 | 0.4232 | 0.298 | 8.9714 |
| 0.4087 | 34.0 | 4692 | 0.4226 | 0.3003 | 8.9796 |
| 0.4087 | 35.0 | 4830 | 0.4224 | 0.2992 | 9.1796 |
| 0.4087 | 36.0 | 4968 | 0.4225 | 0.3005 | 9.0571 |
| 0.4053 | 37.0 | 5106 | 0.4224 | 0.2994 | 8.8571 |
| 0.4053 | 38.0 | 5244 | 0.4220 | 0.3 | 9.1143 |
| 0.4053 | 39.0 | 5382 | 0.4216 | 0.3019 | 9.102 |
| 0.4006 | 40.0 | 5520 | 0.4215 | 0.3016 | 8.9714 |
| 0.4006 | 41.0 | 5658 | 0.4212 | 0.3011 | 8.9224 |
| 0.4006 | 42.0 | 5796 | 0.4211 | 0.2982 | 9.2816 |
| 0.4006 | 43.0 | 5934 | 0.4210 | 0.2985 | 9.1633 |
| 0.3986 | 44.0 | 6072 | 0.4210 | 0.2994 | 9.0776 |
| 0.3986 | 45.0 | 6210 | 0.4209 | 0.308 | 9.3265 |
| 0.3986 | 46.0 | 6348 | 0.4208 | 0.2963 | 9.1714 |
| 0.3986 | 47.0 | 6486 | 0.4205 | 0.3093 | 9.0531 |
| 0.3953 | 48.0 | 6624 | 0.4205 | 0.3068 | 9.4449 |
| 0.3953 | 49.0 | 6762 | 0.4202 | 0.3075 | 8.9918 |
| 0.3953 | 50.0 | 6900 | 0.4203 | 0.3071 | 9.1306 |
| 0.3929 | 51.0 | 7038 | 0.4200 | 0.3052 | 9.3143 |
| 0.3929 | 52.0 | 7176 | 0.4200 | 0.306 | 9.1796 |
| 0.3929 | 53.0 | 7314 | 0.4200 | 0.3058 | 9.2204 |
| 0.3929 | 54.0 | 7452 | 0.4200 | 0.3076 | 8.8367 |
| 0.391 | 55.0 | 7590 | 0.4196 | 0.3078 | 8.8776 |
| 0.391 | 56.0 | 7728 | 0.4197 | 0.3041 | 9.0449 |
| 0.391 | 57.0 | 7866 | 0.4198 | 0.3041 | 8.8776 |
| 0.3887 | 58.0 | 8004 | 0.4201 | 0.3171 | 8.9224 |
| 0.3887 | 59.0 | 8142 | 0.4192 | 0.3074 | 9.0449 |
| 0.3887 | 60.0 | 8280 | 0.4197 | 0.318 | 8.8571 |
| 0.3887 | 61.0 | 8418 | 0.4194 | 0.3167 | 9.1469 |
| 0.3871 | 62.0 | 8556 | 0.4194 | 0.3186 | 8.8612 |
| 0.3871 | 63.0 | 8694 | 0.4192 | 0.3181 | 8.8245 |
| 0.3871 | 64.0 | 8832 | 0.4192 | 0.3178 | 9.0449 |
| 0.3871 | 65.0 | 8970 | 0.4194 | 0.3168 | 8.9673 |
| 0.3849 | 66.0 | 9108 | 0.4191 | 0.3159 | 8.9184 |
| 0.3849 | 67.0 | 9246 | 0.4192 | 0.3191 | 8.7347 |
| 0.3849 | 68.0 | 9384 | 0.4189 | 0.3173 | 8.8367 |
| 0.3841 | 69.0 | 9522 | 0.4189 | 0.3198 | 8.7633 |
| 0.3841 | 70.0 | 9660 | 0.4189 | 0.3168 | 8.9306 |
| 0.3841 | 71.0 | 9798 | 0.4187 | 0.3182 | 8.9837 |
| 0.3841 | 72.0 | 9936 | 0.4191 | 0.3179 | 8.9918 |
| 0.3823 | 73.0 | 10074 | 0.4189 | 0.3173 | 8.951 |
| 0.3823 | 74.0 | 10212 | 0.4188 | 0.3158 | 8.9551 |
| 0.3823 | 75.0 | 10350 | 0.4188 | 0.3184 | 8.9061 |
| 0.3823 | 76.0 | 10488 | 0.4187 | 0.3174 | 8.9347 |
| 0.3809 | 77.0 | 10626 | 0.4186 | 0.2163 | 9.1061 |
| 0.3809 | 78.0 | 10764 | 0.4189 | 0.2173 | 8.8531 |
| 0.3809 | 79.0 | 10902 | 0.4187 | 0.3156 | 9.0776 |
| 0.3798 | 80.0 | 11040 | 0.4187 | 0.3166 | 8.9796 |
| 0.3798 | 81.0 | 11178 | 0.4187 | 0.3172 | 8.9796 |
| 0.3798 | 82.0 | 11316 | 0.4187 | 0.3177 | 9.0 |
| 0.3798 | 83.0 | 11454 | 0.4187 | 0.3167 | 9.0204 |
| 0.3799 | 84.0 | 11592 | 0.4187 | 0.3166 | 8.9837 |
| 0.3799 | 85.0 | 11730 | 0.4187 | 0.3174 | 9.0776 |
| 0.3799 | 86.0 | 11868 | 0.4187 | 0.2174 | 9.1469 |
| 0.3789 | 87.0 | 12006 | 0.4188 | 0.2167 | 8.9143 |
| 0.3789 | 88.0 | 12144 | 0.4187 | 0.2171 | 9.0327 |
| 0.3789 | 89.0 | 12282 | 0.4187 | 0.217 | 9.0531 |
| 0.3789 | 90.0 | 12420 | 0.4186 | 0.3176 | 9.1102 |
| 0.378 | 91.0 | 12558 | 0.4186 | 0.3182 | 9.0531 |
| 0.378 | 92.0 | 12696 | 0.4186 | 0.3186 | 9.1102 |
| 0.378 | 93.0 | 12834 | 0.4187 | 0.2177 | 9.0163 |
| 0.378 | 94.0 | 12972 | 0.4187 | 0.2172 | 9.0204 |
| 0.3768 | 95.0 | 13110 | 0.4186 | 0.2171 | 9.0204 |
| 0.3768 | 96.0 | 13248 | 0.4186 | 0.2171 | 9.0367 |
| 0.3768 | 97.0 | 13386 | 0.4187 | 0.2173 | 8.9959 |
| 0.3769 | 98.0 | 13524 | 0.4187 | 0.2172 | 8.9959 |
| 0.3769 | 99.0 | 13662 | 0.4187 | 0.2172 | 9.0 |
| 0.3769 | 100.0 | 13800 | 0.4187 | 0.2171 | 9.0204 |
Framework versions
- Transformers 4.45.1
- Pytorch 2.4.0
- Datasets 3.0.1
- Tokenizers 0.20.0