beit-base-patch16-224-dmae-va-U5-42

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on Augusto777/dmae-ve-U5 dataset. It achieves the following results on the evaluation set:

Loss: 1.0039
Accuracy: 0.8167

Model description

Model for multiclass detection of age-related macular degeneration.

Intended uses & limitations

Destined to support medical diagnosis.

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 128
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 42

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9	7	1.3471	0.4667
1.6023	1.94	15	1.0873	0.5833
1.1509	2.97	23	0.9948	0.5833
0.826	4.0	31	0.7244	0.7167
0.826	4.9	38	0.5741	0.7333
0.5551	5.94	46	0.6569	0.75
0.3649	6.97	54	0.6322	0.7167
0.2592	8.0	62	0.6994	0.7333
0.2592	8.9	69	0.6590	0.7333
0.1958	9.94	77	0.6846	0.7667
0.1664	10.97	85	0.7166	0.7667
0.1571	12.0	93	0.7842	0.7833
0.1174	12.9	100	0.8465	0.8
0.1174	13.94	108	0.9116	0.7667
0.0956	14.97	116	0.9741	0.75
0.1252	16.0	124	0.7760	0.8
0.0933	16.9	131	0.9424	0.7833
0.0933	17.94	139	1.0445	0.7333
0.1455	18.97	147	0.8525	0.7333
0.1034	20.0	155	0.8222	0.7667
0.0855	20.9	162	0.8991	0.7833
0.0985	21.94	170	0.8955	0.8
0.0985	22.97	178	0.9603	0.7667
0.087	24.0	186	0.9932	0.7833
0.0832	24.9	193	1.0100	0.7833
0.0632	25.94	201	0.9393	0.7667
0.0632	26.97	209	0.9062	0.7833
0.0778	28.0	217	0.9339	0.8
0.0627	28.9	224	1.0039	0.8167
0.0837	29.94	232	1.0636	0.7333
0.0595	30.97	240	1.0424	0.75
0.0595	32.0	248	1.0514	0.8
0.0706	32.9	255	1.0639	0.7833
0.0565	33.94	263	1.0494	0.7667
0.0515	34.97	271	1.0628	0.7667
0.0515	36.0	279	1.1089	0.7667
0.0614	36.9	286	1.0861	0.8
0.0496	37.94	294	1.0713	0.8

Framework versions

Transformers 4.38.2
Pytorch 2.2.1+cu121
Datasets 2.18.0
Tokenizers 0.15.2

Augusto777/beit-base-patch16-224-dmae-va-U5-42