smids_5x_deit_small_rms_001_fold3
This model is a fine-tuned version of facebook/deit-small-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:
- Loss: 0.5638
- Accuracy: 0.7967
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.8408 | 1.0 | 375 | 0.8491 | 0.5383 |
| 0.836 | 2.0 | 750 | 0.8820 | 0.4983 |
| 0.8155 | 3.0 | 1125 | 0.8200 | 0.5917 |
| 0.829 | 4.0 | 1500 | 0.7980 | 0.5933 |
| 0.8032 | 5.0 | 1875 | 0.8027 | 0.5967 |
| 0.8095 | 6.0 | 2250 | 0.7557 | 0.635 |
| 0.7468 | 7.0 | 2625 | 0.7635 | 0.65 |
| 0.7133 | 8.0 | 3000 | 0.7025 | 0.6667 |
| 0.6424 | 9.0 | 3375 | 0.8608 | 0.6467 |
| 0.6511 | 10.0 | 3750 | 0.6834 | 0.6817 |
| 0.6928 | 11.0 | 4125 | 0.7883 | 0.6183 |
| 0.6757 | 12.0 | 4500 | 0.7380 | 0.635 |
| 0.6473 | 13.0 | 4875 | 0.6942 | 0.6633 |
| 0.5828 | 14.0 | 5250 | 0.6863 | 0.7117 |
| 0.5787 | 15.0 | 5625 | 0.6877 | 0.6933 |
| 0.5711 | 16.0 | 6000 | 0.7012 | 0.685 |
| 0.6198 | 17.0 | 6375 | 0.6000 | 0.7183 |
| 0.6331 | 18.0 | 6750 | 0.6316 | 0.7217 |
| 0.5457 | 19.0 | 7125 | 0.6381 | 0.7333 |
| 0.585 | 20.0 | 7500 | 0.6083 | 0.7367 |
| 0.4779 | 21.0 | 7875 | 0.6292 | 0.7 |
| 0.4504 | 22.0 | 8250 | 0.5995 | 0.7533 |
| 0.513 | 23.0 | 8625 | 0.6005 | 0.735 |
| 0.5931 | 24.0 | 9000 | 0.5450 | 0.76 |
| 0.4836 | 25.0 | 9375 | 0.5749 | 0.7517 |
| 0.4981 | 26.0 | 9750 | 0.5577 | 0.77 |
| 0.5035 | 27.0 | 10125 | 0.5452 | 0.7583 |
| 0.4996 | 28.0 | 10500 | 0.5583 | 0.765 |
| 0.4767 | 29.0 | 10875 | 0.5589 | 0.765 |
| 0.4202 | 30.0 | 11250 | 0.5291 | 0.78 |
| 0.4307 | 31.0 | 11625 | 0.5250 | 0.7967 |
| 0.5107 | 32.0 | 12000 | 0.5223 | 0.7917 |
| 0.4923 | 33.0 | 12375 | 0.5101 | 0.7917 |
| 0.4996 | 34.0 | 12750 | 0.5329 | 0.79 |
| 0.3762 | 35.0 | 13125 | 0.5542 | 0.79 |
| 0.4379 | 36.0 | 13500 | 0.5598 | 0.7883 |
| 0.4018 | 37.0 | 13875 | 0.5521 | 0.7983 |
| 0.4033 | 38.0 | 14250 | 0.5506 | 0.7767 |
| 0.4228 | 39.0 | 14625 | 0.5150 | 0.7917 |
| 0.366 | 40.0 | 15000 | 0.5580 | 0.8017 |
| 0.3549 | 41.0 | 15375 | 0.5360 | 0.8067 |
| 0.3677 | 42.0 | 15750 | 0.5521 | 0.8 |
| 0.4255 | 43.0 | 16125 | 0.5412 | 0.8033 |
| 0.355 | 44.0 | 16500 | 0.5640 | 0.7717 |
| 0.3586 | 45.0 | 16875 | 0.5441 | 0.7783 |
| 0.3404 | 46.0 | 17250 | 0.5592 | 0.7867 |
| 0.3867 | 47.0 | 17625 | 0.5593 | 0.8 |
| 0.3586 | 48.0 | 18000 | 0.5571 | 0.8067 |
| 0.2696 | 49.0 | 18375 | 0.5541 | 0.8 |
| 0.3761 | 50.0 | 18750 | 0.5638 | 0.7967 |
Framework versions
- Transformers 4.32.1
- Pytorch 2.1.0+cu121
- Datasets 2.12.0
- Tokenizers 0.13.2