vit-base_rvl-cdip-small_rvl_cdip-NK1000_kd_NKD_t1.0_g1.5_rand

This model is a fine-tuned version of google/vit-base-patch16-224-in21k on the None dataset. It achieves the following results on the evaluation set:

Loss: 6.1637
Accuracy: 0.6275
Brier Loss: 0.6026
Nll: 2.9068
F1 Micro: 0.6275
F1 Macro: 0.6313
Ece: 0.2499
Aurc: 0.1609

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Brier Loss	Nll	F1 Micro	F1 Macro	Ece	Aurc
6.3322	1.0	1000	6.0794	0.1835	0.8928	6.5679	0.1835	0.1322	0.0627	0.6846
5.8198	2.0	2000	5.5963	0.3668	0.7821	3.5543	0.3668	0.3217	0.0967	0.4448
5.53	3.0	3000	5.4184	0.4225	0.7382	3.4217	0.4225	0.3848	0.1087	0.3778
5.3449	4.0	4000	5.1895	0.4655	0.6813	3.0794	0.4655	0.4562	0.1076	0.3029
5.2467	5.0	5000	5.1813	0.4592	0.6845	2.9944	0.4592	0.4430	0.1009	0.3125
5.1382	6.0	6000	5.0102	0.4998	0.6423	2.7804	0.4998	0.4926	0.1013	0.2660
5.0255	7.0	7000	4.9611	0.501	0.6350	2.7692	0.501	0.5085	0.0795	0.2690
4.9089	8.0	8000	4.9327	0.508	0.6204	2.6580	0.508	0.5068	0.0622	0.2565
4.8337	9.0	9000	4.8324	0.5467	0.5866	2.5636	0.5467	0.5419	0.0642	0.2274
4.747	10.0	10000	5.0170	0.5302	0.6080	2.7672	0.5302	0.5193	0.0622	0.2452
4.622	11.0	11000	4.8259	0.5593	0.5709	2.6791	0.5593	0.5520	0.0619	0.2090
4.5449	12.0	12000	4.7696	0.5675	0.5583	2.5273	0.5675	0.5678	0.0541	0.2016
4.447	13.0	13000	4.8718	0.5575	0.5775	2.7597	0.5575	0.5557	0.0575	0.2142
4.341	14.0	14000	4.7644	0.5897	0.5368	2.5797	0.5897	0.5930	0.0560	0.1835
4.2476	15.0	15000	4.8339	0.5905	0.5485	2.6684	0.5905	0.5903	0.0719	0.1872
4.1592	16.0	16000	4.7828	0.5877	0.5456	2.7300	0.5877	0.5877	0.0784	0.1832
4.0513	17.0	17000	4.8771	0.5885	0.5533	2.9097	0.5885	0.5930	0.0965	0.1867
3.9646	18.0	18000	4.8980	0.596	0.5499	2.8383	0.596	0.5948	0.1025	0.1797
3.8768	19.0	19000	4.9787	0.605	0.5551	2.8903	0.605	0.6050	0.1302	0.1765
3.7739	20.0	20000	5.1202	0.5945	0.5727	3.0393	0.5945	0.5935	0.1493	0.1821
3.7023	21.0	21000	5.1879	0.5998	0.5785	2.9570	0.5998	0.5991	0.1690	0.1807
3.6301	22.0	22000	5.2707	0.5933	0.5908	3.1177	0.5933	0.5971	0.1863	0.1829
3.5857	23.0	23000	5.2522	0.5887	0.5994	3.2051	0.5887	0.5949	0.1928	0.1857
3.5256	24.0	24000	5.3443	0.6102	0.5857	2.9687	0.6102	0.6084	0.1953	0.1760
3.4954	25.0	25000	5.3010	0.6045	0.5874	3.0184	0.6045	0.6053	0.1851	0.1807
3.46	26.0	26000	5.4451	0.5992	0.5994	3.0539	0.5992	0.6033	0.2053	0.1819
3.4086	27.0	27000	5.4299	0.608	0.5913	3.1127	0.608	0.6082	0.2027	0.1751
3.3769	28.0	28000	5.6979	0.601	0.6236	3.1077	0.601	0.6024	0.2396	0.1777
3.3238	29.0	29000	5.6090	0.611	0.6013	3.0875	0.611	0.6114	0.2238	0.1729
3.3011	30.0	30000	5.6356	0.6105	0.5991	2.9450	0.6105	0.6123	0.2243	0.1719
3.2708	31.0	31000	5.7634	0.604	0.6181	2.9119	0.604	0.6075	0.2402	0.1771
3.2556	32.0	32000	5.7042	0.617	0.6002	2.9324	0.617	0.6199	0.2263	0.1740
3.2213	33.0	33000	5.7388	0.603	0.6121	2.9240	0.603	0.6108	0.2345	0.1782
3.2138	34.0	34000	5.8008	0.6218	0.6001	2.9209	0.6218	0.6206	0.2284	0.1701
3.1994	35.0	35000	5.7350	0.6142	0.5967	2.9021	0.6142	0.6147	0.2294	0.1688
3.1776	36.0	36000	5.7487	0.609	0.6032	2.8651	0.609	0.6121	0.2329	0.1689
3.1606	37.0	37000	5.8022	0.6165	0.6075	2.8604	0.6165	0.6189	0.2398	0.1677
3.1405	38.0	38000	5.8133	0.6235	0.5949	2.8775	0.6235	0.6272	0.2319	0.1640
3.132	39.0	39000	5.8934	0.6232	0.5974	2.9324	0.6232	0.6274	0.2389	0.1639
3.1303	40.0	40000	5.8902	0.6288	0.5947	2.9049	0.6288	0.6322	0.2344	0.1634
3.1187	41.0	41000	5.9076	0.6215	0.5987	2.8584	0.6215	0.6261	0.2394	0.1630
3.0969	42.0	42000	5.9469	0.6265	0.5984	2.8509	0.6265	0.6309	0.2375	0.1631
3.0964	43.0	43000	5.9442	0.6252	0.5951	2.9309	0.6252	0.6291	0.2397	0.1607
3.0953	44.0	44000	6.0126	0.6238	0.5998	2.8956	0.6238	0.6274	0.2419	0.1630
3.0904	45.0	45000	6.0602	0.6295	0.5991	2.8669	0.6295	0.6334	0.2417	0.1609
3.0794	46.0	46000	6.0782	0.6282	0.6027	2.8830	0.6282	0.6321	0.2442	0.1616
3.0788	47.0	47000	6.1062	0.6275	0.6003	2.8472	0.6275	0.6316	0.2471	0.1610
3.0802	48.0	48000	6.1079	0.6285	0.5998	2.8916	0.6285	0.6322	0.2465	0.1600
3.0644	49.0	49000	6.1569	0.6275	0.6025	2.8941	0.6275	0.6314	0.2497	0.1610
3.0751	50.0	50000	6.1637	0.6275	0.6026	2.9068	0.6275	0.6313	0.2499	0.1609

Framework versions

Transformers 4.26.1
Pytorch 1.13.1.post200
Datasets 2.9.0
Tokenizers 0.13.2

jordyvl/vit-base_rvl-cdip-small_rvl_cdip-NK1000_kd_NKD_t1.0_g1.5_rand

vit-base_rvl-cdip-small_rvl_cdip-NK1000_kd_NKD_t1.0_g1.5_rand

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions