vit-small_tobacco3482_kd_CEKD_t1.5_a0.5

This model is a fine-tuned version of WinKawaks/vit-small-patch16-224 on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.4258
Accuracy: 0.825
Brier Loss: 0.2707
Nll: 0.8867
F1 Micro: 0.825
F1 Macro: 0.8116
Ece: 0.2129
Aurc: 0.0681

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 128
eval_batch_size: 128
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Brier Loss	Nll	F1 Micro	F1 Macro	Ece	Aurc
No log	1.0	7	1.7307	0.22	0.8748	5.3766	0.22	0.1294	0.2444	0.6913
No log	2.0	14	1.3514	0.405	0.7426	3.5573	0.405	0.2280	0.2900	0.4026
No log	3.0	21	0.9121	0.62	0.5647	1.9398	0.62	0.5595	0.2879	0.2015
No log	4.0	28	0.7084	0.695	0.4179	1.7042	0.695	0.6379	0.2305	0.1177
No log	5.0	35	0.7167	0.735	0.3862	1.7929	0.735	0.7392	0.2380	0.1046
No log	6.0	42	0.6442	0.765	0.3625	1.5688	0.765	0.7549	0.2371	0.1034
No log	7.0	49	0.6147	0.805	0.3410	1.5975	0.805	0.7789	0.2438	0.1042
No log	8.0	56	0.6444	0.775	0.3446	1.2309	0.775	0.7725	0.2305	0.0911
No log	9.0	63	0.5964	0.8	0.3219	1.3613	0.8000	0.7784	0.2446	0.0734
No log	10.0	70	0.5700	0.82	0.3160	1.2605	0.82	0.7860	0.2301	0.0632
No log	11.0	77	0.5663	0.79	0.3176	1.2939	0.79	0.7643	0.2315	0.0666
No log	12.0	84	0.5111	0.825	0.3143	1.1082	0.825	0.8082	0.2519	0.0844
No log	13.0	91	0.5228	0.78	0.3156	0.9444	0.78	0.7773	0.1941	0.0650
No log	14.0	98	0.5792	0.78	0.3409	1.5054	0.78	0.7725	0.2061	0.1019
No log	15.0	105	0.4905	0.83	0.2912	1.0068	0.83	0.8266	0.2324	0.0545
No log	16.0	112	0.4990	0.825	0.2961	1.1452	0.825	0.8140	0.2188	0.0632
No log	17.0	119	0.4900	0.805	0.2940	1.2027	0.805	0.8018	0.2188	0.0860
No log	18.0	126	0.4755	0.805	0.2988	1.0223	0.805	0.7789	0.2229	0.0792
No log	19.0	133	0.4398	0.81	0.2679	0.9732	0.81	0.7830	0.2085	0.0585
No log	20.0	140	0.4766	0.805	0.2992	0.9730	0.805	0.7934	0.2141	0.0662
No log	21.0	147	0.4615	0.835	0.2867	0.9343	0.835	0.8219	0.1999	0.0751
No log	22.0	154	0.4343	0.825	0.2641	1.1353	0.825	0.8070	0.2095	0.0603
No log	23.0	161	0.4291	0.85	0.2660	1.0109	0.85	0.8365	0.2435	0.0615
No log	24.0	168	0.4263	0.855	0.2653	0.9395	0.855	0.8440	0.2445	0.0623
No log	25.0	175	0.4338	0.845	0.2700	0.8794	0.845	0.8349	0.2254	0.0584
No log	26.0	182	0.4305	0.835	0.2648	0.9062	0.835	0.8322	0.2113	0.0658
No log	27.0	189	0.4262	0.84	0.2683	0.9967	0.8400	0.8291	0.2240	0.0670
No log	28.0	196	0.4329	0.83	0.2724	0.9016	0.83	0.8239	0.2016	0.0685
No log	29.0	203	0.4233	0.845	0.2653	0.9115	0.845	0.8375	0.2005	0.0634
No log	30.0	210	0.4204	0.84	0.2638	0.8892	0.8400	0.8348	0.2175	0.0633
No log	31.0	217	0.4240	0.83	0.2684	0.8871	0.83	0.8217	0.2128	0.0660
No log	32.0	224	0.4246	0.84	0.2677	0.8867	0.8400	0.8307	0.2117	0.0670
No log	33.0	231	0.4247	0.83	0.2690	0.8917	0.83	0.8202	0.2084	0.0679
No log	34.0	238	0.4218	0.84	0.2660	0.8848	0.8400	0.8326	0.2138	0.0663
No log	35.0	245	0.4220	0.845	0.2667	0.8926	0.845	0.8354	0.2109	0.0655
No log	36.0	252	0.4247	0.83	0.2694	0.8854	0.83	0.8202	0.2213	0.0683
No log	37.0	259	0.4239	0.84	0.2683	0.8849	0.8400	0.8326	0.2163	0.0670
No log	38.0	266	0.4239	0.835	0.2689	0.8876	0.835	0.8208	0.2118	0.0672
No log	39.0	273	0.4252	0.83	0.2696	0.8885	0.83	0.8180	0.2064	0.0682
No log	40.0	280	0.4237	0.835	0.2686	0.8867	0.835	0.8208	0.2211	0.0675
No log	41.0	287	0.4256	0.83	0.2700	0.8847	0.83	0.8180	0.2253	0.0682
No log	42.0	294	0.4243	0.835	0.2692	0.8839	0.835	0.8208	0.2130	0.0675
No log	43.0	301	0.4248	0.83	0.2695	0.8850	0.83	0.8180	0.2237	0.0682
No log	44.0	308	0.4246	0.83	0.2694	0.8847	0.83	0.8180	0.2383	0.0680
No log	45.0	315	0.4253	0.83	0.2699	0.8858	0.83	0.8180	0.2200	0.0681
No log	46.0	322	0.4246	0.83	0.2694	0.8857	0.83	0.8180	0.2311	0.0679
No log	47.0	329	0.4253	0.83	0.2700	0.8843	0.83	0.8180	0.2312	0.0682
No log	48.0	336	0.4252	0.83	0.2698	0.8830	0.83	0.8180	0.2177	0.0682
No log	49.0	343	0.4257	0.83	0.2703	0.8848	0.83	0.8180	0.2315	0.0683
No log	50.0	350	0.4256	0.83	0.2703	0.8833	0.83	0.8180	0.2331	0.0684
No log	51.0	357	0.4254	0.83	0.2703	0.8863	0.83	0.8180	0.2422	0.0681
No log	52.0	364	0.4261	0.83	0.2707	0.8864	0.83	0.8180	0.2424	0.0683
No log	53.0	371	0.4249	0.83	0.2700	0.8855	0.83	0.8180	0.2195	0.0679
No log	54.0	378	0.4255	0.83	0.2704	0.8846	0.83	0.8180	0.2342	0.0682
No log	55.0	385	0.4256	0.825	0.2704	0.8861	0.825	0.8116	0.2357	0.0682
No log	56.0	392	0.4264	0.83	0.2708	0.8853	0.83	0.8180	0.2345	0.0682
No log	57.0	399	0.4257	0.825	0.2706	0.8864	0.825	0.8116	0.2353	0.0682
No log	58.0	406	0.4258	0.825	0.2704	0.8841	0.825	0.8116	0.2271	0.0681
No log	59.0	413	0.4255	0.825	0.2703	0.8856	0.825	0.8116	0.2267	0.0680
No log	60.0	420	0.4259	0.825	0.2709	0.8842	0.825	0.8116	0.2269	0.0683
No log	61.0	427	0.4254	0.83	0.2702	0.8852	0.83	0.8180	0.2265	0.0680
No log	62.0	434	0.4261	0.83	0.2707	0.8851	0.83	0.8180	0.2346	0.0682
No log	63.0	441	0.4257	0.825	0.2704	0.8854	0.825	0.8116	0.2232	0.0682
No log	64.0	448	0.4261	0.825	0.2708	0.8845	0.825	0.8116	0.2264	0.0683
No log	65.0	455	0.4259	0.825	0.2706	0.8862	0.825	0.8116	0.2204	0.0682
No log	66.0	462	0.4258	0.825	0.2707	0.8856	0.825	0.8116	0.2193	0.0682
No log	67.0	469	0.4255	0.83	0.2703	0.8852	0.83	0.8180	0.2190	0.0681
No log	68.0	476	0.4260	0.825	0.2708	0.8860	0.825	0.8116	0.2196	0.0682
No log	69.0	483	0.4259	0.825	0.2708	0.8858	0.825	0.8116	0.2195	0.0682
No log	70.0	490	0.4255	0.825	0.2703	0.8857	0.825	0.8116	0.2135	0.0682
No log	71.0	497	0.4258	0.825	0.2707	0.8857	0.825	0.8116	0.2205	0.0681
0.1816	72.0	504	0.4261	0.825	0.2708	0.8857	0.825	0.8116	0.2198	0.0682
0.1816	73.0	511	0.4259	0.825	0.2706	0.8852	0.825	0.8116	0.2192	0.0682
0.1816	74.0	518	0.4259	0.825	0.2707	0.8856	0.825	0.8116	0.2290	0.0681
0.1816	75.0	525	0.4257	0.825	0.2706	0.8864	0.825	0.8116	0.2337	0.0681
0.1816	76.0	532	0.4259	0.825	0.2707	0.8855	0.825	0.8116	0.2211	0.0681
0.1816	77.0	539	0.4255	0.825	0.2704	0.8860	0.825	0.8116	0.2137	0.0680
0.1816	78.0	546	0.4258	0.825	0.2707	0.8868	0.825	0.8116	0.2274	0.0682
0.1816	79.0	553	0.4260	0.825	0.2708	0.8859	0.825	0.8116	0.2209	0.0682
0.1816	80.0	560	0.4260	0.825	0.2708	0.8864	0.825	0.8116	0.2135	0.0681
0.1816	81.0	567	0.4259	0.825	0.2707	0.8859	0.825	0.8116	0.2134	0.0682
0.1816	82.0	574	0.4258	0.825	0.2706	0.8862	0.825	0.8116	0.2062	0.0681
0.1816	83.0	581	0.4259	0.825	0.2707	0.8866	0.825	0.8116	0.2204	0.0681
0.1816	84.0	588	0.4259	0.825	0.2707	0.8868	0.825	0.8116	0.2204	0.0681
0.1816	85.0	595	0.4257	0.825	0.2706	0.8861	0.825	0.8116	0.2141	0.0682
0.1816	86.0	602	0.4258	0.825	0.2707	0.8861	0.825	0.8116	0.2140	0.0682
0.1816	87.0	609	0.4258	0.825	0.2707	0.8867	0.825	0.8116	0.2137	0.0680
0.1816	88.0	616	0.4259	0.825	0.2707	0.8866	0.825	0.8116	0.2129	0.0681
0.1816	89.0	623	0.4258	0.825	0.2707	0.8866	0.825	0.8116	0.2205	0.0681
0.1816	90.0	630	0.4259	0.825	0.2707	0.8865	0.825	0.8116	0.2053	0.0680
0.1816	91.0	637	0.4258	0.825	0.2706	0.8868	0.825	0.8116	0.2130	0.0681
0.1816	92.0	644	0.4258	0.825	0.2706	0.8870	0.825	0.8116	0.2129	0.0680
0.1816	93.0	651	0.4258	0.825	0.2706	0.8868	0.825	0.8116	0.2129	0.0681
0.1816	94.0	658	0.4258	0.825	0.2707	0.8867	0.825	0.8116	0.2129	0.0681
0.1816	95.0	665	0.4258	0.825	0.2707	0.8867	0.825	0.8116	0.2053	0.0680
0.1816	96.0	672	0.4259	0.825	0.2707	0.8866	0.825	0.8116	0.2053	0.0681
0.1816	97.0	679	0.4258	0.825	0.2707	0.8868	0.825	0.8116	0.2129	0.0681
0.1816	98.0	686	0.4258	0.825	0.2707	0.8868	0.825	0.8116	0.2129	0.0680
0.1816	99.0	693	0.4258	0.825	0.2707	0.8868	0.825	0.8116	0.2129	0.0681
0.1816	100.0	700	0.4258	0.825	0.2707	0.8867	0.825	0.8116	0.2129	0.0681

Framework versions

Transformers 4.26.1
Pytorch 1.13.1.post200
Datasets 2.9.0
Tokenizers 0.13.2

jordyvl/vit-small_tobacco3482_kd_CEKD_t1.5_a0.5

vit-small_tobacco3482_kd_CEKD_t1.5_a0.5

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions