vit-base_rvl-cdip-small_rvl_cdip-NK1000_kd_CEKD_t2.5_a0.5_rand

This model is a fine-tuned version of google/vit-base-patch16-224-in21k on the None dataset. It achieves the following results on the evaluation set:

Loss: 1.9130
Accuracy: 0.6115
Brier Loss: 0.5924
Nll: 2.8591
F1 Micro: 0.6115
F1 Macro: 0.6137
Ece: 0.2201
Aurc: 0.1762

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Brier Loss	Nll	F1 Micro	F1 Macro	Ece	Aurc
4.0737	1.0	1000	3.8362	0.1948	0.8785	4.2153	0.1948	0.1339	0.0877	0.6770
3.2534	2.0	2000	3.0129	0.3772	0.7858	3.2453	0.3773	0.3171	0.1532	0.4263
2.802	3.0	3000	2.6716	0.4472	0.7236	3.0740	0.4472	0.4150	0.1479	0.3537
2.5803	4.0	4000	2.4882	0.4775	0.6581	2.9147	0.4775	0.4658	0.0774	0.2981
2.4445	5.0	5000	2.3888	0.4753	0.6741	2.8504	0.4753	0.4562	0.1529	0.2912
2.2315	6.0	6000	2.2439	0.5132	0.6404	2.7536	0.5132	0.5007	0.1179	0.2659
2.0849	7.0	7000	2.2140	0.5125	0.6441	2.8160	0.5125	0.5119	0.1522	0.2634
1.922	8.0	8000	2.0941	0.5298	0.6202	2.7843	0.5298	0.5267	0.1511	0.2384
1.8235	9.0	9000	2.0477	0.5453	0.5974	2.6273	0.5453	0.5505	0.1131	0.2252
1.6864	10.0	10000	2.0429	0.5575	0.6181	2.7309	0.5575	0.5391	0.1828	0.2172
1.5498	11.0	11000	2.0547	0.5597	0.6272	2.6772	0.5597	0.5444	0.1931	0.2237
1.4502	12.0	12000	2.0086	0.5707	0.6175	2.7419	0.5707	0.5592	0.1935	0.2153
1.3331	13.0	13000	2.0399	0.566	0.6150	2.8462	0.566	0.5660	0.1786	0.2175
1.19	14.0	14000	2.0207	0.5805	0.6209	2.7599	0.5805	0.5793	0.2133	0.2081
1.0615	15.0	15000	2.0089	0.5793	0.6224	2.9014	0.5793	0.5787	0.2157	0.2077
0.9517	16.0	16000	2.0644	0.5765	0.6300	2.9717	0.5765	0.5796	0.2264	0.2047
0.8343	17.0	17000	2.0990	0.5745	0.6473	3.0444	0.5745	0.5712	0.2502	0.2050
0.7191	18.0	18000	2.1704	0.5763	0.6527	3.0809	0.5763	0.5721	0.2539	0.2125
0.638	19.0	19000	2.1930	0.5807	0.6572	3.1618	0.5807	0.5792	0.2620	0.2120
0.5528	20.0	20000	2.1731	0.5885	0.6542	3.1412	0.5885	0.5898	0.2627	0.2067
0.4957	21.0	21000	2.2492	0.5763	0.6708	3.1701	0.5763	0.5760	0.2700	0.2232
0.4096	22.0	22000	2.3164	0.5707	0.6837	3.3874	0.5707	0.5706	0.2761	0.2260
0.3915	23.0	23000	2.3277	0.58	0.6813	3.4230	0.58	0.5814	0.2857	0.2165
0.3412	24.0	24000	2.2947	0.5813	0.6779	3.3373	0.5813	0.5854	0.2870	0.2076
0.3171	25.0	25000	2.2743	0.586	0.6720	3.3310	0.586	0.5848	0.2767	0.2104
0.2734	26.0	26000	2.2762	0.593	0.6696	3.3439	0.593	0.5977	0.2819	0.2044
0.2535	27.0	27000	2.2205	0.5845	0.6605	3.2712	0.5845	0.5864	0.2684	0.2060
0.229	28.0	28000	2.2961	0.5845	0.6821	3.2738	0.5845	0.5902	0.2896	0.2106
0.2202	29.0	29000	2.2698	0.5845	0.6752	3.2127	0.5845	0.5845	0.2835	0.2128
0.1922	30.0	30000	2.2511	0.5787	0.6731	3.3305	0.5787	0.5819	0.2770	0.2065
0.1857	31.0	31000	2.1847	0.5863	0.6598	3.2211	0.5863	0.5889	0.2692	0.2036
0.1678	32.0	32000	2.1752	0.5913	0.6551	3.0691	0.5913	0.5926	0.2680	0.2010
0.1587	33.0	33000	2.1107	0.5972	0.6392	3.0412	0.5972	0.6002	0.2495	0.2002
0.1432	34.0	34000	2.2079	0.5893	0.6593	3.1988	0.5893	0.5917	0.2640	0.2092
0.1291	35.0	35000	2.0788	0.592	0.6388	2.9891	0.592	0.5942	0.2587	0.1887
0.1245	36.0	36000	2.0521	0.601	0.6297	2.9432	0.601	0.6018	0.2464	0.1923
0.1169	37.0	37000	2.0669	0.5935	0.6294	3.0364	0.5935	0.5932	0.2425	0.1878
0.1068	38.0	38000	2.0197	0.606	0.6183	2.9448	0.606	0.6054	0.2375	0.1897
0.1031	39.0	39000	2.0022	0.6032	0.6163	2.8697	0.6032	0.6051	0.2429	0.1880
0.0961	40.0	40000	1.9965	0.6058	0.6147	2.9143	0.6058	0.6091	0.2393	0.1859
0.093	41.0	41000	1.9735	0.6082	0.6080	2.9346	0.6082	0.6100	0.2331	0.1834
0.0859	42.0	42000	1.9383	0.6092	0.6011	2.8692	0.6092	0.6105	0.2281	0.1792
0.0817	43.0	43000	1.9413	0.6115	0.5999	2.8908	0.6115	0.6124	0.2264	0.1802
0.0779	44.0	44000	1.9271	0.6112	0.6001	2.8695	0.6112	0.6127	0.2274	0.1791
0.0745	45.0	45000	1.9259	0.6135	0.5983	2.8467	0.6135	0.6154	0.2206	0.1786
0.0723	46.0	46000	1.9214	0.612	0.5964	2.8289	0.612	0.6136	0.2233	0.1778
0.0673	47.0	47000	1.9117	0.6128	0.5923	2.8226	0.6128	0.6148	0.2197	0.1768
0.0649	48.0	48000	1.9111	0.6152	0.5911	2.8575	0.6152	0.6164	0.2166	0.1757
0.0651	49.0	49000	1.9106	0.6148	0.5914	2.8675	0.6148	0.6169	0.2164	0.1761
0.0629	50.0	50000	1.9130	0.6115	0.5924	2.8591	0.6115	0.6137	0.2201	0.1762

Framework versions

Transformers 4.26.1
Pytorch 1.13.1.post200
Datasets 2.9.0
Tokenizers 0.13.2

jordyvl/vit-base_rvl-cdip-small_rvl_cdip-NK1000_kd_CEKD_t2.5_a0.5_rand

vit-base_rvl-cdip-small_rvl_cdip-NK1000_kd_CEKD_t2.5_a0.5_rand

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions