swin-ena24

This model is a fine-tuned version of microsoft/swin-base-simmim-window6-192 on the ena24 dataset. It achieves the following results on the evaluation set:

Loss: 2.4677
Accuracy: 0.5146
F1 Macro: 0.4328

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 7
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Macro
1.9888	0.2519	100	3.4122	0.1631	0.0893
1.6111	0.5038	200	2.9077	0.2578	0.1535
1.1276	0.7557	300	2.6504	0.3574	0.2827
1.0234	1.0076	400	2.5728	0.3906	0.3156
0.8909	1.2594	500	2.5007	0.4219	0.3388
0.8008	1.5113	600	2.7039	0.4043	0.3619
0.6885	1.7632	700	3.1090	0.3701	0.2926
0.839	2.0151	800	2.5845	0.4844	0.4149
0.325	2.2670	900	2.5143	0.5068	0.4128
0.4501	2.5189	1000	2.7684	0.4482	0.4056
0.3191	2.7708	1100	2.4677	0.5146	0.4328
0.1664	3.0227	1200	2.4777	0.5361	0.4597
0.1469	3.2746	1300	2.6403	0.5205	0.4495
0.3063	3.5264	1400	2.8000	0.5010	0.4415
0.1786	3.7783	1500	2.8165	0.5332	0.4525
0.0687	4.0302	1600	2.9027	0.5684	0.4942
0.0427	4.2821	1700	3.3216	0.4912	0.4362
0.1825	4.5340	1800	3.1456	0.5312	0.4664
0.0758	4.7859	1900	3.2782	0.5547	0.4578
0.0471	5.0378	2000	3.3348	0.5518	0.4725
0.0512	5.2897	2100	3.7182	0.5283	0.4514
0.0095	5.5416	2200	3.9028	0.5342	0.4785
0.0247	5.7935	2300	3.9606	0.5713	0.4879
0.0008	6.0453	2400	4.1290	0.5654	0.4918
0.0024	6.2972	2500	4.4147	0.5654	0.4863
0.0002	6.5491	2600	4.5209	0.5654	0.4913
0.0055	6.8010	2700	4.5154	0.5820	0.5067

Framework versions

Transformers 4.51.3
Pytorch 2.6.0+cu124
Datasets 3.6.0
Tokenizers 0.21.1

Loading…

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002

train_batch_size: 16

eval_batch_size: 16

seed: 42

optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments

lr_scheduler_type: linear

num_epochs: 7

mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Macro
1.9888	0.2519	100	3.4122	0.1631	0.0893
1.6111	0.5038	200	2.9077	0.2578	0.1535
1.1276	0.7557	300	2.6504	0.3574	0.2827
1.0234	1.0076	400	2.5728	0.3906	0.3156
0.8909	1.2594	500	2.5007	0.4219	0.3388
0.8008	1.5113	600	2.7039	0.4043	0.3619
0.6885	1.7632	700	3.1090	0.3701	0.2926
0.839	2.0151	800	2.5845	0.4844	0.4149
0.325	2.2670	900	2.5143	0.5068	0.4128
0.4501	2.5189	1000	2.7684	0.4482	0.4056
0.3191	2.7708	1100	2.4677	0.5146	0.4328
0.1664	3.0227	1200	2.4777	0.5361	0.4597
0.1469	3.2746	1300	2.6403	0.5205	0.4495
0.3063	3.5264	1400	2.8000	0.5010	0.4415
0.1786	3.7783	1500	2.8165	0.5332	0.4525
0.0687	4.0302	1600	2.9027	0.5684	0.4942
0.0427	4.2821	1700	3.3216	0.4912	0.4362
0.1825	4.5340	1800	3.1456	0.5312	0.4664
0.0758	4.7859	1900	3.2782	0.5547	0.4578
0.0471	5.0378	2000	3.3348	0.5518	0.4725
0.0512	5.2897	2100	3.7182	0.5283	0.4514
0.0095	5.5416	2200	3.9028	0.5342	0.4785
0.0247	5.7935	2300	3.9606	0.5713	0.4879
0.0008	6.0453	2400	4.1290	0.5654	0.4918
0.0024	6.2972	2500	4.4147	0.5654	0.4863
0.0002	6.5491	2600	4.5209	0.5654	0.4913
0.0055	6.8010	2700	4.5154	0.5820	0.5067

Framework versions

Transformers 4.51.3

Pytorch 2.6.0+cu124

Datasets 3.6.0

Tokenizers 0.21.1

Pamreth/swin-ena24

swin-ena24

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

swin-ena24

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions