t5-small-t5-dialogue-summarizer
This model is a fine-tuned version of t5-small on the samsum dataset. It achieves the following results on the evaluation set:
- Loss: 1.7032
- Rouge1: 43.3371
- Rouge2: 20.6294
- Rougel: 36.6607
- Rougelsum: 40.209
- Gen Len: 16.698
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 460 | 1.8115 | 41.2589 | 18.3552 | 34.5107 | 38.2488 | 16.8068 |
| 1.9846 | 2.0 | 921 | 1.7892 | 41.1617 | 18.4345 | 34.745 | 38.2061 | 16.6247 |
| 1.9568 | 3.0 | 1381 | 1.7757 | 41.7317 | 19.0104 | 35.2965 | 38.6958 | 16.4059 |
| 1.9298 | 4.0 | 1842 | 1.7573 | 42.0478 | 19.1229 | 35.4855 | 39.0882 | 16.6235 |
| 1.9049 | 5.0 | 2302 | 1.7496 | 42.4985 | 19.5594 | 35.9228 | 39.4201 | 16.5416 |
| 1.8852 | 6.0 | 2763 | 1.7411 | 42.3214 | 19.6152 | 35.7488 | 39.3079 | 16.7139 |
| 1.8674 | 7.0 | 3223 | 1.7335 | 42.3206 | 19.7528 | 35.9918 | 39.2783 | 16.5073 |
| 1.855 | 8.0 | 3684 | 1.7300 | 42.9099 | 20.2273 | 36.4393 | 39.8506 | 16.61 |
| 1.8435 | 9.0 | 4144 | 1.7225 | 42.9661 | 20.3074 | 36.3468 | 39.8945 | 16.7103 |
| 1.8342 | 10.0 | 4605 | 1.7198 | 43.0181 | 20.2982 | 36.4202 | 39.9022 | 16.7726 |
| 1.8216 | 11.0 | 5065 | 1.7169 | 43.0296 | 20.5422 | 36.6314 | 40.111 | 16.6883 |
| 1.8168 | 12.0 | 5526 | 1.7144 | 43.3035 | 20.7167 | 36.7924 | 40.2953 | 16.7787 |
| 1.8168 | 13.0 | 5986 | 1.7104 | 43.2258 | 20.7416 | 36.7823 | 40.2551 | 16.7286 |
| 1.8088 | 14.0 | 6447 | 1.7075 | 43.3982 | 20.8281 | 36.8254 | 40.3198 | 16.7384 |
| 1.8008 | 15.0 | 6907 | 1.7079 | 43.3077 | 20.7164 | 36.6791 | 40.2372 | 16.687 |
| 1.8014 | 16.0 | 7368 | 1.7047 | 43.1989 | 20.6984 | 36.7104 | 40.2285 | 16.6479 |
| 1.7934 | 17.0 | 7828 | 1.7034 | 43.4149 | 20.7879 | 36.7308 | 40.3556 | 16.7922 |
| 1.7894 | 18.0 | 8289 | 1.7041 | 43.2962 | 20.7667 | 36.7017 | 40.28 | 16.6883 |
| 1.7914 | 19.0 | 8749 | 1.7037 | 43.2489 | 20.6943 | 36.676 | 40.1802 | 16.6932 |
| 1.7827 | 19.98 | 9200 | 1.7032 | 43.3371 | 20.6294 | 36.6607 | 40.209 | 16.698 |
Framework versions
- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0