small-vanilla-target-conll2003
This model is a fine-tuned version of google/bert_uncased_L-4_H-512_A-8 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.1128
- Precision: 0.8947
- Recall: 0.9165
- F1: 0.9055
- Accuracy: 0.9782
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- training_steps: 5000
Training results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
|---|---|---|---|---|---|---|---|
| 0.217 | 1.14 | 500 | 0.0895 | 0.8414 | 0.8846 | 0.8624 | 0.9725 |
| 0.0723 | 2.28 | 1000 | 0.0808 | 0.8853 | 0.8975 | 0.8914 | 0.9762 |
| 0.0427 | 3.42 | 1500 | 0.0807 | 0.8886 | 0.9118 | 0.9001 | 0.9778 |
| 0.0274 | 4.56 | 2000 | 0.0816 | 0.8983 | 0.9142 | 0.9062 | 0.9790 |
| 0.0186 | 5.69 | 2500 | 0.0910 | 0.8990 | 0.9201 | 0.9094 | 0.9788 |
| 0.0132 | 6.83 | 3000 | 0.0933 | 0.8978 | 0.9180 | 0.9078 | 0.9789 |
| 0.0097 | 7.97 | 3500 | 0.0954 | 0.9025 | 0.9211 | 0.9117 | 0.9798 |
| 0.0076 | 9.11 | 4000 | 0.1003 | 0.9007 | 0.9221 | 0.9113 | 0.9794 |
| 0.0058 | 10.25 | 4500 | 0.1037 | 0.9032 | 0.9222 | 0.9126 | 0.9798 |
| 0.005 | 11.39 | 5000 | 0.1128 | 0.8947 | 0.9165 | 0.9055 | 0.9782 |
Framework versions
- Transformers 4.26.0.dev0
- Pytorch 1.13.0+cu116
- Datasets 2.8.1.dev0
- Tokenizers 0.13.2