dit-base-finetuned-rvlcdip-small_rvl_cdip-NK1000_kd_NKD_t1.0_g1.5
This model is a fine-tuned version of WinKawaks/vit-small-patch16-224 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 5.3085
- Accuracy: 0.828
- Brier Loss: 0.3005
- Nll: 1.3339
- F1 Micro: 0.828
- F1 Macro: 0.8285
- Ece: 0.1391
- Aurc: 0.0474
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 96
- eval_batch_size: 96
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll | F1 Micro | F1 Macro | Ece | Aurc |
|---|---|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 167 | 5.3750 | 0.61 | 0.5591 | 2.2520 | 0.61 | 0.5922 | 0.1827 | 0.1664 |
| No log | 2.0 | 334 | 5.0343 | 0.7117 | 0.4389 | 1.9483 | 0.7117 | 0.7096 | 0.1691 | 0.0962 |
| 5.4927 | 3.0 | 501 | 4.8554 | 0.7472 | 0.3777 | 1.6689 | 0.7472 | 0.7474 | 0.1221 | 0.0780 |
| 5.4927 | 4.0 | 668 | 4.7917 | 0.76 | 0.3524 | 1.5715 | 0.76 | 0.7644 | 0.0915 | 0.0699 |
| 5.4927 | 5.0 | 835 | 4.7792 | 0.765 | 0.3461 | 1.5348 | 0.765 | 0.7590 | 0.0737 | 0.0704 |
| 4.6216 | 6.0 | 1002 | 4.6378 | 0.7993 | 0.2954 | 1.3769 | 0.7993 | 0.7995 | 0.0546 | 0.0559 |
| 4.6216 | 7.0 | 1169 | 4.8666 | 0.771 | 0.3359 | 1.5727 | 0.771 | 0.7728 | 0.0670 | 0.0666 |
| 4.6216 | 8.0 | 1336 | 4.6834 | 0.7897 | 0.3047 | 1.3537 | 0.7897 | 0.7914 | 0.0531 | 0.0564 |
| 4.2978 | 9.0 | 1503 | 4.6558 | 0.7997 | 0.2912 | 1.3758 | 0.7997 | 0.7988 | 0.0521 | 0.0508 |
| 4.2978 | 10.0 | 1670 | 4.8214 | 0.7923 | 0.3144 | 1.5316 | 0.7923 | 0.7928 | 0.0815 | 0.0561 |
| 4.2978 | 11.0 | 1837 | 4.8908 | 0.7923 | 0.3201 | 1.4158 | 0.7923 | 0.7931 | 0.0988 | 0.0573 |
| 4.1375 | 12.0 | 2004 | 4.7703 | 0.8093 | 0.2971 | 1.3642 | 0.8093 | 0.8097 | 0.0812 | 0.0514 |
| 4.1375 | 13.0 | 2171 | 4.8126 | 0.806 | 0.3039 | 1.3759 | 0.806 | 0.8053 | 0.0916 | 0.0491 |
| 4.1375 | 14.0 | 2338 | 4.7875 | 0.8063 | 0.2990 | 1.3712 | 0.8062 | 0.8065 | 0.0904 | 0.0481 |
| 4.0665 | 15.0 | 2505 | 4.7995 | 0.805 | 0.3016 | 1.4133 | 0.805 | 0.8049 | 0.0909 | 0.0503 |
| 4.0665 | 16.0 | 2672 | 4.7712 | 0.8075 | 0.2957 | 1.4018 | 0.8075 | 0.8082 | 0.0880 | 0.0484 |
| 4.0665 | 17.0 | 2839 | 4.7245 | 0.812 | 0.2886 | 1.2816 | 0.8120 | 0.8139 | 0.0831 | 0.0464 |
| 4.0204 | 18.0 | 3006 | 4.8990 | 0.8055 | 0.3079 | 1.3884 | 0.8055 | 0.8046 | 0.1117 | 0.0504 |
| 4.0204 | 19.0 | 3173 | 4.9286 | 0.802 | 0.3147 | 1.3977 | 0.802 | 0.7995 | 0.1078 | 0.0522 |
| 4.0204 | 20.0 | 3340 | 4.9510 | 0.8055 | 0.3121 | 1.4482 | 0.8055 | 0.8062 | 0.1125 | 0.0521 |
| 3.9854 | 21.0 | 3507 | 4.8837 | 0.8033 | 0.3082 | 1.4528 | 0.8033 | 0.8022 | 0.1052 | 0.0502 |
| 3.9854 | 22.0 | 3674 | 5.0103 | 0.813 | 0.3069 | 1.4217 | 0.813 | 0.8169 | 0.1207 | 0.0500 |
| 3.9854 | 23.0 | 3841 | 4.9602 | 0.8093 | 0.3091 | 1.4672 | 0.8093 | 0.8103 | 0.1187 | 0.0494 |
| 3.9599 | 24.0 | 4008 | 4.8980 | 0.8177 | 0.2953 | 1.3589 | 0.8178 | 0.8203 | 0.1083 | 0.0451 |
| 3.9599 | 25.0 | 4175 | 4.8753 | 0.8145 | 0.2932 | 1.3219 | 0.8145 | 0.8140 | 0.1054 | 0.0460 |
| 3.9599 | 26.0 | 4342 | 4.9644 | 0.8193 | 0.3000 | 1.4336 | 0.8193 | 0.8200 | 0.1173 | 0.0458 |
| 3.9358 | 27.0 | 4509 | 4.9648 | 0.8203 | 0.2985 | 1.4117 | 0.8203 | 0.8197 | 0.1132 | 0.0471 |
| 3.9358 | 28.0 | 4676 | 5.0130 | 0.8195 | 0.3014 | 1.4618 | 0.8195 | 0.8201 | 0.1205 | 0.0456 |
| 3.9358 | 29.0 | 4843 | 4.8974 | 0.8255 | 0.2874 | 1.3041 | 0.8255 | 0.8258 | 0.1097 | 0.0421 |
| 3.9151 | 30.0 | 5010 | 4.9045 | 0.8255 | 0.2878 | 1.2801 | 0.8255 | 0.8250 | 0.1119 | 0.0426 |
| 3.9151 | 31.0 | 5177 | 4.9720 | 0.823 | 0.2945 | 1.3551 | 0.823 | 0.8240 | 0.1212 | 0.0439 |
| 3.9151 | 32.0 | 5344 | 4.9508 | 0.826 | 0.2913 | 1.2669 | 0.826 | 0.8268 | 0.1201 | 0.0422 |
| 3.9003 | 33.0 | 5511 | 5.0336 | 0.8237 | 0.2991 | 1.3443 | 0.8237 | 0.8240 | 0.1243 | 0.0433 |
| 3.9003 | 34.0 | 5678 | 4.9828 | 0.8237 | 0.2901 | 1.2843 | 0.8237 | 0.8239 | 0.1214 | 0.0440 |
| 3.9003 | 35.0 | 5845 | 5.0256 | 0.8287 | 0.2920 | 1.2961 | 0.8287 | 0.8291 | 0.1232 | 0.0422 |
| 3.89 | 36.0 | 6012 | 5.0229 | 0.8283 | 0.2922 | 1.2471 | 0.8283 | 0.8283 | 0.1236 | 0.0432 |
| 3.89 | 37.0 | 6179 | 5.0835 | 0.8285 | 0.2936 | 1.2892 | 0.8285 | 0.8289 | 0.1254 | 0.0442 |
| 3.89 | 38.0 | 6346 | 5.1148 | 0.8287 | 0.2946 | 1.3106 | 0.8287 | 0.8282 | 0.1287 | 0.0427 |
| 3.8846 | 39.0 | 6513 | 5.1238 | 0.827 | 0.2954 | 1.2964 | 0.827 | 0.8275 | 0.1298 | 0.0441 |
| 3.8846 | 40.0 | 6680 | 5.1481 | 0.8307 | 0.2950 | 1.3136 | 0.8308 | 0.8311 | 0.1277 | 0.0453 |
| 3.8846 | 41.0 | 6847 | 5.1491 | 0.8293 | 0.2943 | 1.2841 | 0.8293 | 0.8294 | 0.1298 | 0.0451 |
| 3.881 | 42.0 | 7014 | 5.1982 | 0.829 | 0.2969 | 1.3111 | 0.8290 | 0.8292 | 0.1331 | 0.0459 |
| 3.881 | 43.0 | 7181 | 5.2041 | 0.8283 | 0.2970 | 1.3427 | 0.8283 | 0.8283 | 0.1327 | 0.0465 |
| 3.881 | 44.0 | 7348 | 5.2310 | 0.8297 | 0.2985 | 1.3351 | 0.8297 | 0.8303 | 0.1346 | 0.0471 |
| 3.8796 | 45.0 | 7515 | 5.2394 | 0.83 | 0.2999 | 1.3308 | 0.83 | 0.8305 | 0.1348 | 0.0467 |
| 3.8796 | 46.0 | 7682 | 5.2632 | 0.83 | 0.2990 | 1.3350 | 0.83 | 0.8304 | 0.1355 | 0.0471 |
| 3.8796 | 47.0 | 7849 | 5.2821 | 0.828 | 0.2998 | 1.3354 | 0.828 | 0.8286 | 0.1383 | 0.0470 |
| 3.8753 | 48.0 | 8016 | 5.2949 | 0.829 | 0.2998 | 1.3341 | 0.8290 | 0.8294 | 0.1374 | 0.0472 |
| 3.8753 | 49.0 | 8183 | 5.3026 | 0.8287 | 0.3004 | 1.3281 | 0.8287 | 0.8293 | 0.1382 | 0.0474 |
| 3.8753 | 50.0 | 8350 | 5.3085 | 0.828 | 0.3005 | 1.3339 | 0.828 | 0.8285 | 0.1391 | 0.0474 |
Framework versions
- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2