deberta-v3-xsmall-survey-rater-sample-1
This model is a fine-tuned version of microsoft/deberta-v3-xsmall on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.7323
- F1: 0.7588
- Precision: 0.7588
- Recall: 0.7588
- Krippendorff: 0.7739
- Krippendorff Swap: 0.0020
- Accuracy: 0.7588
- Absolute Agreement: 0.7588
- Agreement Within One: 0.9585
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 6e-06
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1000
- num_epochs: 20
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | F1 | Precision | Recall | Krippendorff | Krippendorff Swap | Accuracy | Absolute Agreement | Agreement Within One |
|---|---|---|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 199 | 1.9615 | 0.1667 | 0.1667 | 0.1667 | -0.6139 | 0.6225 | 0.1667 | 0.1667 | 1.0 |
| No log | 2.0 | 398 | 1.9276 | 0.1667 | 0.1667 | 0.1667 | -0.6139 | 0.6225 | 0.1667 | 0.1667 | 1.0 |
| 1.7267 | 3.0 | 597 | 2.0600 | 0.1667 | 0.1667 | 0.1667 | -0.6139 | 0.6225 | 0.1667 | 0.1667 | 1.0 |
| 1.7267 | 4.0 | 796 | 2.1002 | 0.1667 | 0.1667 | 0.1667 | -0.6139 | 0.6225 | 0.1667 | 0.1667 | 1.0 |
| 1.7267 | 5.0 | 995 | 2.2144 | 0.1667 | 0.1667 | 0.1667 | -0.6139 | 0.6225 | 0.1667 | 0.1667 | 1.0 |
| 1.2295 | 6.0 | 1194 | 2.3125 | 0.1667 | 0.1667 | 0.1667 | -0.6139 | 0.6225 | 0.1667 | 0.1667 | 1.0 |
| 1.2295 | 7.0 | 1393 | 2.3633 | 0.2639 | 0.2639 | 0.2639 | -0.1509 | 0.0619 | 0.2639 | 0.2639 | 0.7917 |
| 1.1344 | 8.0 | 1592 | 2.5500 | 0.1667 | 0.1667 | 0.1667 | -0.6139 | 0.6225 | 0.1667 | 0.1667 | 1.0 |
| 1.1344 | 9.0 | 1791 | 2.4821 | 0.25 | 0.25 | 0.25 | -0.1730 | 0.0619 | 0.25 | 0.25 | 0.7917 |
| 1.1344 | 10.0 | 1990 | 2.5046 | 0.2639 | 0.2639 | 0.2639 | -0.1187 | 0.0344 | 0.2639 | 0.2639 | 0.7778 |
| 1.0895 | 11.0 | 2189 | 2.3104 | 0.3194 | 0.3194 | 0.3194 | -0.2512 | 0.0511 | 0.3194 | 0.3194 | 0.6528 |
| 1.0895 | 12.0 | 2388 | 2.4332 | 0.2917 | 0.2917 | 0.2917 | -0.0433 | 0.0783 | 0.2917 | 0.2917 | 0.8472 |
| 1.0113 | 13.0 | 2587 | 2.4675 | 0.3611 | 0.3611 | 0.3611 | 0.2089 | -0.0044 | 0.3611 | 0.3611 | 0.8056 |
| 1.0113 | 14.0 | 2786 | 2.3801 | 0.3611 | 0.3611 | 0.3611 | 0.2432 | 0.0344 | 0.3611 | 0.3611 | 0.8889 |
| 1.0113 | 15.0 | 2985 | 2.2074 | 0.4306 | 0.4306 | 0.4306 | 0.4808 | -0.0002 | 0.4306 | 0.4306 | 0.8889 |
| 0.9084 | 16.0 | 3184 | 2.2794 | 0.4306 | 0.4306 | 0.4306 | 0.4622 | 0.0232 | 0.4306 | 0.4306 | 0.9167 |
| 0.9084 | 17.0 | 3383 | 2.1940 | 0.4583 | 0.4583 | 0.4583 | 0.5431 | -0.0044 | 0.4583 | 0.4583 | 0.8889 |
| 0.8226 | 18.0 | 3582 | 2.2878 | 0.4306 | 0.4306 | 0.4306 | 0.5002 | 0.0136 | 0.4306 | 0.4306 | 0.9167 |
| 0.8226 | 19.0 | 3781 | 2.2772 | 0.4306 | 0.4306 | 0.4306 | 0.5002 | 0.0136 | 0.4306 | 0.4306 | 0.9167 |
| 0.8226 | 20.0 | 3980 | 2.2638 | 0.4306 | 0.4306 | 0.4306 | 0.5002 | 0.0136 | 0.4306 | 0.4306 | 0.9167 |
Framework versions
- Transformers 4.18.0
- Pytorch 1.11.0
- Datasets 2.1.0
- Tokenizers 0.12.1