vit-base-patch16-224-in21k-tiny_rvl_cdip-NK1000_hint

This model is a fine-tuned version of WinKawaks/vit-tiny-patch16-224 on the None dataset. It achieves the following results on the evaluation set:

Loss: 75.3146
Accuracy: 0.811
Brier Loss: 0.3395
Nll: 2.0856
F1 Micro: 0.811
F1 Macro: 0.8109
Ece: 0.1625
Aurc: 0.0586

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Brier Loss	Nll	F1 Micro	F1 Macro	Ece	Aurc
78.9278	1.0	1000	78.3645	0.588	0.5476	2.4320	0.588	0.5825	0.0552	0.1928
77.6109	2.0	2000	77.4909	0.684	0.4259	2.1686	0.684	0.6879	0.0496	0.1167
77.1849	3.0	3000	77.1828	0.7185	0.3854	2.1032	0.7185	0.7200	0.0495	0.0932
76.8526	4.0	4000	76.9800	0.748	0.3549	2.0716	0.748	0.7492	0.0768	0.0781
76.5928	5.0	5000	76.7544	0.743	0.3576	2.0634	0.743	0.7461	0.0564	0.0846
76.1507	6.0	6000	76.5850	0.7688	0.3354	2.0506	0.7688	0.7698	0.0857	0.0701
75.8107	7.0	7000	76.5816	0.75	0.3766	2.1753	0.75	0.7542	0.1230	0.0815
75.868	8.0	8000	76.5048	0.785	0.3324	2.0865	0.785	0.7869	0.1244	0.0623
75.6016	9.0	9000	76.3919	0.7827	0.3475	2.0858	0.7828	0.7849	0.1340	0.0657
75.4883	10.0	10000	76.5121	0.7768	0.3644	2.1372	0.7768	0.7747	0.1550	0.0662
75.2568	11.0	11000	76.4107	0.7857	0.3603	2.1229	0.7857	0.7863	0.1561	0.0619
75.1623	12.0	12000	76.4517	0.771	0.3857	2.1118	0.771	0.7721	0.1681	0.0702
75.0021	13.0	13000	76.3632	0.7885	0.3635	2.1178	0.7885	0.7870	0.1607	0.0621
74.9056	14.0	14000	76.3074	0.7925	0.3533	2.1361	0.7925	0.7926	0.1626	0.0573
74.9295	15.0	15000	76.3445	0.785	0.3730	2.0515	0.785	0.7861	0.1694	0.0661
74.7288	16.0	16000	76.3441	0.7845	0.3776	2.1216	0.7845	0.7828	0.1731	0.0666
74.5985	17.0	17000	76.1255	0.794	0.3593	2.0759	0.7940	0.7969	0.1640	0.0605
74.471	18.0	18000	76.2140	0.7863	0.3721	2.1872	0.7863	0.7861	0.1705	0.0638
74.4457	19.0	19000	76.1380	0.7925	0.3650	2.1106	0.7925	0.7940	0.1708	0.0634
74.3675	20.0	20000	76.1423	0.7897	0.3684	2.0882	0.7897	0.7910	0.1731	0.0642
74.3618	21.0	21000	76.0578	0.7987	0.3604	2.1007	0.7987	0.7982	0.1676	0.0622
74.1398	22.0	22000	75.9928	0.7997	0.3578	2.0590	0.7997	0.8008	0.1672	0.0624
74.0834	23.0	23000	75.8857	0.8013	0.3561	2.0986	0.8013	0.8010	0.1662	0.0602
74.1467	24.0	24000	75.8767	0.8	0.3605	2.0794	0.8000	0.8014	0.1682	0.0608
73.8823	25.0	25000	75.9471	0.799	0.3564	2.0934	0.799	0.7997	0.1684	0.0619
73.9657	26.0	26000	75.8618	0.7987	0.3594	2.1020	0.7987	0.7991	0.1703	0.0599
73.9721	27.0	27000	75.7331	0.8145	0.3347	2.0514	0.8145	0.8144	0.1569	0.0571
73.8298	28.0	28000	75.8175	0.8007	0.3582	2.0923	0.8007	0.7999	0.1714	0.0625
73.8483	29.0	29000	75.7541	0.8023	0.3554	2.1075	0.8023	0.8002	0.1698	0.0603
73.6726	30.0	30000	75.6642	0.8095	0.3454	2.0600	0.8095	0.8092	0.1638	0.0600
73.7118	31.0	31000	75.5905	0.8105	0.3398	2.1354	0.8105	0.8106	0.1587	0.0595
73.5938	32.0	32000	75.5721	0.8087	0.3429	2.0765	0.8087	0.8094	0.1640	0.0616
73.5563	33.0	33000	75.7021	0.8085	0.3474	2.0825	0.8085	0.8092	0.1656	0.0633
73.6469	34.0	34000	75.5322	0.8095	0.3406	2.0907	0.8095	0.8079	0.1632	0.0590
73.4666	35.0	35000	75.4994	0.8105	0.3397	2.0839	0.8105	0.8102	0.1621	0.0590
73.4144	36.0	36000	75.5095	0.8063	0.3476	2.1055	0.8062	0.8050	0.1666	0.0616
73.2744	37.0	37000	75.4980	0.8117	0.3403	2.0693	0.8117	0.8123	0.1607	0.0569
73.4358	38.0	38000	75.4824	0.809	0.3434	2.0996	0.809	0.8090	0.1645	0.0586
73.2696	39.0	39000	75.5088	0.8085	0.3468	2.0697	0.8085	0.8089	0.1658	0.0589
73.382	40.0	40000	75.4705	0.8095	0.3437	2.0738	0.8095	0.8104	0.1641	0.0621
73.3006	41.0	41000	75.4697	0.809	0.3440	2.1203	0.809	0.8097	0.1624	0.0614
73.4237	42.0	42000	75.3601	0.8093	0.3434	2.0736	0.8093	0.8094	0.1629	0.0575
73.2571	43.0	43000	75.3364	0.8103	0.3398	2.0665	0.8103	0.8101	0.1630	0.0599
73.2241	44.0	44000	75.3369	0.8135	0.3381	2.0609	0.8135	0.8136	0.1600	0.0581
73.2271	45.0	45000	75.2917	0.814	0.3355	2.0906	0.8140	0.8142	0.1576	0.0582
73.1427	46.0	46000	75.3108	0.8125	0.3377	2.0784	0.8125	0.8127	0.1613	0.0586
73.2754	47.0	47000	75.3195	0.8127	0.3386	2.0860	0.8128	0.8127	0.1604	0.0586
73.132	48.0	48000	75.3168	0.812	0.3391	2.0853	0.8120	0.8118	0.1612	0.0582
73.1482	49.0	49000	75.2943	0.8117	0.3395	2.0895	0.8117	0.8116	0.1615	0.0586
73.1849	50.0	50000	75.3146	0.811	0.3395	2.0856	0.811	0.8109	0.1625	0.0586

Framework versions

Transformers 4.26.1
Pytorch 1.13.1.post200
Datasets 2.9.0
Tokenizers 0.13.2

加载中…

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001

train_batch_size: 16

eval_batch_size: 16

seed: 42

optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08

lr_scheduler_type: linear

lr_scheduler_warmup_ratio: 0.1

num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Brier Loss	Nll	F1 Micro	F1 Macro	Ece	Aurc
78.9278	1.0	1000	78.3645	0.588	0.5476	2.4320	0.588	0.5825	0.0552	0.1928
77.6109	2.0	2000	77.4909	0.684	0.4259	2.1686	0.684	0.6879	0.0496	0.1167
77.1849	3.0	3000	77.1828	0.7185	0.3854	2.1032	0.7185	0.7200	0.0495	0.0932
76.8526	4.0	4000	76.9800	0.748	0.3549	2.0716	0.748	0.7492	0.0768	0.0781
76.5928	5.0	5000	76.7544	0.743	0.3576	2.0634	0.743	0.7461	0.0564	0.0846
76.1507	6.0	6000	76.5850	0.7688	0.3354	2.0506	0.7688	0.7698	0.0857	0.0701
75.8107	7.0	7000	76.5816	0.75	0.3766	2.1753	0.75	0.7542	0.1230	0.0815
75.868	8.0	8000	76.5048	0.785	0.3324	2.0865	0.785	0.7869	0.1244	0.0623
75.6016	9.0	9000	76.3919	0.7827	0.3475	2.0858	0.7828	0.7849	0.1340	0.0657
75.4883	10.0	10000	76.5121	0.7768	0.3644	2.1372	0.7768	0.7747	0.1550	0.0662
75.2568	11.0	11000	76.4107	0.7857	0.3603	2.1229	0.7857	0.7863	0.1561	0.0619
75.1623	12.0	12000	76.4517	0.771	0.3857	2.1118	0.771	0.7721	0.1681	0.0702
75.0021	13.0	13000	76.3632	0.7885	0.3635	2.1178	0.7885	0.7870	0.1607	0.0621
74.9056	14.0	14000	76.3074	0.7925	0.3533	2.1361	0.7925	0.7926	0.1626	0.0573
74.9295	15.0	15000	76.3445	0.785	0.3730	2.0515	0.785	0.7861	0.1694	0.0661
74.7288	16.0	16000	76.3441	0.7845	0.3776	2.1216	0.7845	0.7828	0.1731	0.0666
74.5985	17.0	17000	76.1255	0.794	0.3593	2.0759	0.7940	0.7969	0.1640	0.0605
74.471	18.0	18000	76.2140	0.7863	0.3721	2.1872	0.7863	0.7861	0.1705	0.0638
74.4457	19.0	19000	76.1380	0.7925	0.3650	2.1106	0.7925	0.7940	0.1708	0.0634
74.3675	20.0	20000	76.1423	0.7897	0.3684	2.0882	0.7897	0.7910	0.1731	0.0642
74.3618	21.0	21000	76.0578	0.7987	0.3604	2.1007	0.7987	0.7982	0.1676	0.0622
74.1398	22.0	22000	75.9928	0.7997	0.3578	2.0590	0.7997	0.8008	0.1672	0.0624
74.0834	23.0	23000	75.8857	0.8013	0.3561	2.0986	0.8013	0.8010	0.1662	0.0602
74.1467	24.0	24000	75.8767	0.8	0.3605	2.0794	0.8000	0.8014	0.1682	0.0608
73.8823	25.0	25000	75.9471	0.799	0.3564	2.0934	0.799	0.7997	0.1684	0.0619
73.9657	26.0	26000	75.8618	0.7987	0.3594	2.1020	0.7987	0.7991	0.1703	0.0599
73.9721	27.0	27000	75.7331	0.8145	0.3347	2.0514	0.8145	0.8144	0.1569	0.0571
73.8298	28.0	28000	75.8175	0.8007	0.3582	2.0923	0.8007	0.7999	0.1714	0.0625
73.8483	29.0	29000	75.7541	0.8023	0.3554	2.1075	0.8023	0.8002	0.1698	0.0603
73.6726	30.0	30000	75.6642	0.8095	0.3454	2.0600	0.8095	0.8092	0.1638	0.0600
73.7118	31.0	31000	75.5905	0.8105	0.3398	2.1354	0.8105	0.8106	0.1587	0.0595
73.5938	32.0	32000	75.5721	0.8087	0.3429	2.0765	0.8087	0.8094	0.1640	0.0616
73.5563	33.0	33000	75.7021	0.8085	0.3474	2.0825	0.8085	0.8092	0.1656	0.0633
73.6469	34.0	34000	75.5322	0.8095	0.3406	2.0907	0.8095	0.8079	0.1632	0.0590
73.4666	35.0	35000	75.4994	0.8105	0.3397	2.0839	0.8105	0.8102	0.1621	0.0590
73.4144	36.0	36000	75.5095	0.8063	0.3476	2.1055	0.8062	0.8050	0.1666	0.0616
73.2744	37.0	37000	75.4980	0.8117	0.3403	2.0693	0.8117	0.8123	0.1607	0.0569
73.4358	38.0	38000	75.4824	0.809	0.3434	2.0996	0.809	0.8090	0.1645	0.0586
73.2696	39.0	39000	75.5088	0.8085	0.3468	2.0697	0.8085	0.8089	0.1658	0.0589
73.382	40.0	40000	75.4705	0.8095	0.3437	2.0738	0.8095	0.8104	0.1641	0.0621
73.3006	41.0	41000	75.4697	0.809	0.3440	2.1203	0.809	0.8097	0.1624	0.0614
73.4237	42.0	42000	75.3601	0.8093	0.3434	2.0736	0.8093	0.8094	0.1629	0.0575
73.2571	43.0	43000	75.3364	0.8103	0.3398	2.0665	0.8103	0.8101	0.1630	0.0599
73.2241	44.0	44000	75.3369	0.8135	0.3381	2.0609	0.8135	0.8136	0.1600	0.0581
73.2271	45.0	45000	75.2917	0.814	0.3355	2.0906	0.8140	0.8142	0.1576	0.0582
73.1427	46.0	46000	75.3108	0.8125	0.3377	2.0784	0.8125	0.8127	0.1613	0.0586
73.2754	47.0	47000	75.3195	0.8127	0.3386	2.0860	0.8128	0.8127	0.1604	0.0586
73.132	48.0	48000	75.3168	0.812	0.3391	2.0853	0.8120	0.8118	0.1612	0.0582
73.1482	49.0	49000	75.2943	0.8117	0.3395	2.0895	0.8117	0.8116	0.1615	0.0586
73.1849	50.0	50000	75.3146	0.811	0.3395	2.0856	0.811	0.8109	0.1625	0.0586

Framework versions

Transformers 4.26.1

Pytorch 1.13.1.post200

Datasets 2.9.0

Tokenizers 0.13.2

jordyvl/vit-base-patch16-224-in21k-tiny_rvl_cdip-NK1000_hint

vit-base-patch16-224-in21k-tiny_rvl_cdip-NK1000_hint

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

vit-base-patch16-224-in21k-tiny_rvl_cdip-NK1000_hint

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions