fine_tuned_t5_small_model_sec_5_v5
This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:
- Loss: 2.8447
- Rouge1: 0.3701
- Rouge2: 0.1378
- Rougel: 0.2427
- Rougelsum: 0.2427
- Gen Len: 78.0105
- Bert F1: 0.8699
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 5
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len | Bert F1 |
|---|---|---|---|---|---|---|---|---|---|
| 3.8604 | 0.4211 | 40 | 3.2077 | 0.3522 | 0.1312 | 0.2314 | 0.2321 | 60.8684 | 0.8715 |
| 3.4406 | 0.8421 | 80 | 3.0222 | 0.3615 | 0.1333 | 0.2327 | 0.2331 | 70.9421 | 0.8725 |
| 3.2592 | 1.2632 | 120 | 2.9491 | 0.3796 | 0.1401 | 0.2392 | 0.2395 | 81.9421 | 0.8728 |
| 3.1817 | 1.6842 | 160 | 2.9082 | 0.3791 | 0.135 | 0.2382 | 0.2383 | 83.7579 | 0.873 |
| 3.1808 | 2.1053 | 200 | 2.8893 | 0.3817 | 0.1403 | 0.2433 | 0.2436 | 82.9211 | 0.8741 |
| 3.1333 | 2.5263 | 240 | 2.8745 | 0.3737 | 0.1365 | 0.2427 | 0.2431 | 80.8632 | 0.8735 |
| 3.1758 | 2.9474 | 280 | 2.8623 | 0.3751 | 0.142 | 0.245 | 0.2452 | 79.6526 | 0.8744 |
| 3.0898 | 3.3684 | 320 | 2.8559 | 0.3739 | 0.1407 | 0.2441 | 0.2443 | 80.1684 | 0.8741 |
| 3.1227 | 3.7895 | 360 | 2.8499 | 0.3739 | 0.1406 | 0.2458 | 0.2456 | 78.8789 | 0.8743 |
| 3.0641 | 4.2105 | 400 | 2.8467 | 0.3678 | 0.1368 | 0.2418 | 0.2418 | 78.2053 | 0.8691 |
| 3.0768 | 4.6316 | 440 | 2.8447 | 0.3701 | 0.1378 | 0.2427 | 0.2427 | 78.0105 | 0.8699 |
Framework versions
- Transformers 4.46.3
- Pytorch 2.4.0
- Datasets 3.1.0
- Tokenizers 0.20.3