smids_10x_deit_small_sgd_0001_fold3
This model is a fine-tuned version of facebook/deit-small-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:
- Loss: 0.3901
- Accuracy: 0.855
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.9824 | 1.0 | 750 | 1.0331 | 0.435 |
| 0.9063 | 2.0 | 1500 | 0.9735 | 0.5233 |
| 0.8503 | 3.0 | 2250 | 0.9109 | 0.59 |
| 0.7679 | 4.0 | 3000 | 0.8466 | 0.645 |
| 0.7248 | 5.0 | 3750 | 0.7860 | 0.69 |
| 0.6585 | 6.0 | 4500 | 0.7311 | 0.7167 |
| 0.6129 | 7.0 | 5250 | 0.6856 | 0.7283 |
| 0.6082 | 8.0 | 6000 | 0.6417 | 0.7617 |
| 0.581 | 9.0 | 6750 | 0.6068 | 0.7683 |
| 0.5231 | 10.0 | 7500 | 0.5777 | 0.7767 |
| 0.5113 | 11.0 | 8250 | 0.5554 | 0.7833 |
| 0.4834 | 12.0 | 9000 | 0.5347 | 0.8 |
| 0.5002 | 13.0 | 9750 | 0.5194 | 0.8067 |
| 0.5244 | 14.0 | 10500 | 0.5049 | 0.8117 |
| 0.478 | 15.0 | 11250 | 0.4926 | 0.8183 |
| 0.4573 | 16.0 | 12000 | 0.4823 | 0.8183 |
| 0.4332 | 17.0 | 12750 | 0.4737 | 0.8233 |
| 0.4552 | 18.0 | 13500 | 0.4642 | 0.8283 |
| 0.4717 | 19.0 | 14250 | 0.4573 | 0.8283 |
| 0.4284 | 20.0 | 15000 | 0.4511 | 0.8283 |
| 0.418 | 21.0 | 15750 | 0.4442 | 0.835 |
| 0.4355 | 22.0 | 16500 | 0.4394 | 0.8417 |
| 0.442 | 23.0 | 17250 | 0.4349 | 0.84 |
| 0.4592 | 24.0 | 18000 | 0.4307 | 0.845 |
| 0.4174 | 25.0 | 18750 | 0.4266 | 0.8483 |
| 0.4133 | 26.0 | 19500 | 0.4227 | 0.8483 |
| 0.3538 | 27.0 | 20250 | 0.4190 | 0.8517 |
| 0.4061 | 28.0 | 21000 | 0.4159 | 0.8533 |
| 0.4077 | 29.0 | 21750 | 0.4132 | 0.8517 |
| 0.4051 | 30.0 | 22500 | 0.4109 | 0.8533 |
| 0.3404 | 31.0 | 23250 | 0.4086 | 0.8517 |
| 0.353 | 32.0 | 24000 | 0.4061 | 0.855 |
| 0.3864 | 33.0 | 24750 | 0.4039 | 0.8567 |
| 0.3572 | 34.0 | 25500 | 0.4020 | 0.8567 |
| 0.3431 | 35.0 | 26250 | 0.4002 | 0.8567 |
| 0.3693 | 36.0 | 27000 | 0.3992 | 0.8567 |
| 0.3706 | 37.0 | 27750 | 0.3978 | 0.8567 |
| 0.423 | 38.0 | 28500 | 0.3964 | 0.855 |
| 0.3909 | 39.0 | 29250 | 0.3953 | 0.855 |
| 0.41 | 40.0 | 30000 | 0.3943 | 0.855 |
| 0.3387 | 41.0 | 30750 | 0.3933 | 0.855 |
| 0.3698 | 42.0 | 31500 | 0.3927 | 0.855 |
| 0.3644 | 43.0 | 32250 | 0.3919 | 0.855 |
| 0.3722 | 44.0 | 33000 | 0.3914 | 0.8567 |
| 0.3269 | 45.0 | 33750 | 0.3910 | 0.8567 |
| 0.3532 | 46.0 | 34500 | 0.3906 | 0.8567 |
| 0.3899 | 47.0 | 35250 | 0.3904 | 0.8567 |
| 0.3783 | 48.0 | 36000 | 0.3902 | 0.855 |
| 0.3767 | 49.0 | 36750 | 0.3901 | 0.855 |
| 0.3232 | 50.0 | 37500 | 0.3901 | 0.855 |
Framework versions
- Transformers 4.32.1
- Pytorch 2.1.0+cu121
- Datasets 2.12.0
- Tokenizers 0.13.2