t5-small-finetuned-2048

This model is a fine-tuned version of google-t5/t5-small on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 13.3433
Rouge1: 0.029
Rouge2: 0.0023
Rougel: 0.0267
Rougelsum: 0.0284

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
No log	0.67	1	25.1883	0.0242	0.0023	0.0218	0.0241
No log	2.0	3	23.4392	0.0242	0.0023	0.0218	0.0241
No log	2.67	4	22.5166	0.0252	0.0023	0.0229	0.0251
No log	4.0	6	20.6643	0.0252	0.0023	0.0229	0.0251
No log	4.67	7	19.7334	0.0252	0.0023	0.0229	0.0251
No log	6.0	9	17.8137	0.0252	0.0023	0.0229	0.0251
No log	6.67	10	17.1117	0.0252	0.0023	0.0229	0.0251
No log	8.0	12	16.4384	0.0329	0.005	0.0269	0.0324
No log	8.67	13	16.2401	0.0329	0.005	0.0269	0.0324
No log	10.0	15	15.9056	0.0329	0.005	0.0269	0.0324
No log	10.67	16	15.7547	0.0329	0.005	0.0269	0.0324
No log	12.0	18	15.4599	0.0329	0.005	0.0269	0.0324
No log	12.67	19	15.3192	0.0329	0.005	0.0269	0.0324
17.3983	14.0	21	15.0513	0.0329	0.005	0.0269	0.0324
17.3983	14.67	22	14.9270	0.0367	0.005	0.0307	0.0357
17.3983	16.0	24	14.7037	0.0367	0.005	0.0307	0.0357
17.3983	16.67	25	14.5987	0.0367	0.005	0.0307	0.0357
17.3983	18.0	27	14.4010	0.0367	0.005	0.0307	0.0357
17.3983	18.67	28	14.3084	0.0367	0.005	0.0307	0.0357
17.3983	20.0	30	14.1348	0.0367	0.005	0.0307	0.0357
17.3983	20.67	31	14.0554	0.0367	0.005	0.0307	0.0357
17.3983	22.0	33	13.9103	0.0367	0.005	0.0307	0.0357
17.3983	22.67	34	13.8446	0.029	0.0023	0.0267	0.0284
17.3983	24.0	36	13.7251	0.029	0.0023	0.0267	0.0284
17.3983	24.67	37	13.6713	0.029	0.0023	0.0267	0.0284
17.3983	26.0	39	13.5781	0.029	0.0023	0.0267	0.0284
13.2153	26.67	40	13.5376	0.029	0.0023	0.0267	0.0284
13.2153	28.0	42	13.4689	0.029	0.0023	0.0267	0.0284
13.2153	28.67	43	13.4408	0.029	0.0023	0.0267	0.0284
13.2153	30.0	45	13.3953	0.029	0.0023	0.0267	0.0284
13.2153	30.67	46	13.3780	0.029	0.0023	0.0267	0.0284
13.2153	32.0	48	13.3538	0.029	0.0023	0.0267	0.0284
13.2153	32.67	49	13.3468	0.029	0.0023	0.0267	0.0284
13.2153	33.33	50	13.3433	0.029	0.0023	0.0267	0.0284

Framework versions

Transformers 4.38.0.dev0
Pytorch 2.2.0
Datasets 2.16.1
Tokenizers 0.15.1

RMWeerasinghe/t5-small-finetuned-2048

t5-small-finetuned-2048

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

为此模型做种