xlm-roberta-largeFullTune
This model is a fine-tuned version of xlm-roberta-large on SIRIS-Lab/citation-parser-ENTITY dataset. It achieves the following results on the evaluation set:
- Loss: 0.1011
- Precision: 0.9721
- Recall: 0.9708
- F1: 0.9715
- Accuracy: 0.9838
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
|---|---|---|---|---|---|---|---|
| 1.1963 | 1.0 | 68 | 0.1978 | 0.7166 | 0.8617 | 0.7825 | 0.9549 |
| 0.1574 | 2.0 | 136 | 0.1378 | 0.8734 | 0.9363 | 0.9038 | 0.9751 |
| 0.0544 | 3.0 | 204 | 0.1184 | 0.9561 | 0.9666 | 0.9613 | 0.9815 |
| 0.0442 | 4.0 | 272 | 0.1188 | 0.9654 | 0.9691 | 0.9673 | 0.9819 |
| 0.0331 | 5.0 | 340 | 0.1121 | 0.9666 | 0.9691 | 0.9679 | 0.9822 |
| 0.0208 | 6.0 | 408 | 0.1237 | 0.9711 | 0.9736 | 0.9724 | 0.9811 |
| 0.0214 | 7.0 | 476 | 0.1213 | 0.9680 | 0.9730 | 0.9705 | 0.9825 |
| 0.0172 | 8.0 | 544 | 0.1295 | 0.9730 | 0.9736 | 0.9733 | 0.9830 |
| 0.0097 | 9.0 | 612 | 0.1236 | 0.9723 | 0.9717 | 0.9720 | 0.9833 |
| 0.0117 | 10.0 | 680 | 0.1300 | 0.9717 | 0.9730 | 0.9724 | 0.9829 |
Framework versions
- Transformers 5.15.0
- Pytorch 2.11.0+cu128
- Datasets 4.0.0
- Tokenizers 0.22.2