fine_tuned_t5_small_model_sec_5_v3
This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:
- Loss: 2.8693
- Rouge1: 0.3608
- Rouge2: 0.1224
- Rougel: 0.2319
- Rougelsum: 0.2314
- Gen Len: 76.4017
- Bert F1: 0.8703
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len | Bert F1 |
|---|---|---|---|---|---|---|---|---|---|
| 3.8703 | 0.6667 | 20 | 3.4203 | 0.3378 | 0.1167 | 0.2214 | 0.2212 | 57.735 | 0.8692 |
| 3.5213 | 1.3333 | 40 | 3.1803 | 0.35 | 0.1227 | 0.2305 | 0.2297 | 65.5641 | 0.8704 |
| 3.389 | 2.0 | 60 | 3.0624 | 0.3577 | 0.1269 | 0.2295 | 0.2287 | 70.7949 | 0.8696 |
| 3.2528 | 2.6667 | 80 | 2.9966 | 0.3707 | 0.1327 | 0.2402 | 0.2393 | 74.8034 | 0.8707 |
| 3.2518 | 3.3333 | 100 | 2.9522 | 0.3644 | 0.1239 | 0.2318 | 0.2315 | 78.1368 | 0.869 |
| 3.1403 | 4.0 | 120 | 2.9272 | 0.3675 | 0.1246 | 0.2325 | 0.2326 | 77.1538 | 0.8701 |
| 3.1359 | 4.6667 | 140 | 2.9114 | 0.3702 | 0.1294 | 0.2335 | 0.2331 | 78.7692 | 0.8703 |
| 3.1541 | 5.3333 | 160 | 2.8971 | 0.3701 | 0.1276 | 0.2341 | 0.2341 | 79.3846 | 0.8702 |
| 3.1552 | 6.0 | 180 | 2.8869 | 0.3686 | 0.1254 | 0.2328 | 0.2325 | 79.3932 | 0.8703 |
| 3.1116 | 6.6667 | 200 | 2.8805 | 0.3618 | 0.1243 | 0.2326 | 0.2319 | 77.188 | 0.8697 |
| 3.0692 | 7.3333 | 220 | 2.8766 | 0.3646 | 0.1259 | 0.2337 | 0.2331 | 77.5556 | 0.8704 |
| 2.9932 | 8.0 | 240 | 2.8726 | 0.3633 | 0.1244 | 0.234 | 0.2336 | 76.812 | 0.8703 |
| 3.067 | 8.6667 | 260 | 2.8708 | 0.3608 | 0.1234 | 0.2323 | 0.2319 | 76.0769 | 0.8702 |
| 3.0864 | 9.3333 | 280 | 2.8693 | 0.3608 | 0.1224 | 0.2319 | 0.2314 | 76.4017 | 0.8703 |
Framework versions
- Transformers 4.46.3
- Pytorch 2.4.0
- Datasets 3.1.0
- Tokenizers 0.20.3