bart-mlm-paraphrasing
This model is a fine-tuned version of gayanin/bart-mlm-pubmed on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.4617
- Rouge2 Precision: 0.8361
- Rouge2 Recall: 0.6703
- Rouge2 Fmeasure: 0.7304
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge2 Precision | Rouge2 Recall | Rouge2 Fmeasure |
|---|---|---|---|---|---|---|
| 0.4845 | 1.0 | 1325 | 0.4270 | 0.8332 | 0.6701 | 0.7294 |
| 0.3911 | 2.0 | 2650 | 0.4195 | 0.8358 | 0.6713 | 0.7313 |
| 0.328 | 3.0 | 3975 | 0.4119 | 0.8355 | 0.6706 | 0.7304 |
| 0.2783 | 4.0 | 5300 | 0.4160 | 0.8347 | 0.6678 | 0.7284 |
| 0.2397 | 5.0 | 6625 | 0.4329 | 0.8411 | 0.6747 | 0.7351 |
| 0.2155 | 6.0 | 7950 | 0.4389 | 0.8382 | 0.6716 | 0.7321 |
| 0.1888 | 7.0 | 9275 | 0.4432 | 0.838 | 0.6718 | 0.7323 |
| 0.1724 | 8.0 | 10600 | 0.4496 | 0.8381 | 0.6714 | 0.7319 |
| 0.1586 | 9.0 | 11925 | 0.4575 | 0.8359 | 0.6704 | 0.7303 |
| 0.1496 | 10.0 | 13250 | 0.4617 | 0.8361 | 0.6703 | 0.7304 |
Framework versions
- Transformers 4.17.0
- Pytorch 1.10.0+cu111
- Datasets 1.18.4
- Tokenizers 0.11.6