vit-base
This model is a fine-tuned version of google/vit-base-patch16-224-in21k on the cifar100 dataset. It achieves the following results on the evaluation set:
- Loss: 0.3142
- Accuracy: 0.9197
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 128
- eval_batch_size: 256
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 4.0904 | 1.0 | 333 | 4.0142 | 0.6663 |
| 3.5092 | 2.0 | 666 | 3.3563 | 0.7659 |
| 3.0987 | 3.0 | 999 | 2.9060 | 0.8043 |
| 2.7858 | 4.0 | 1332 | 2.5428 | 0.827 |
| 2.4372 | 5.0 | 1665 | 2.2353 | 0.8428 |
| 2.2157 | 6.0 | 1998 | 1.9597 | 0.8568 |
| 1.9573 | 7.0 | 2331 | 1.7241 | 0.8685 |
| 1.841 | 8.0 | 2664 | 1.5210 | 0.8736 |
| 1.6085 | 9.0 | 2997 | 1.3363 | 0.8832 |
| 1.4188 | 10.0 | 3330 | 1.1857 | 0.8861 |
| 1.3424 | 11.0 | 3663 | 1.0522 | 0.8923 |
| 1.1924 | 12.0 | 3996 | 0.9380 | 0.8983 |
| 1.1764 | 13.0 | 4329 | 0.8405 | 0.8999 |
| 1.0548 | 14.0 | 4662 | 0.7641 | 0.9024 |
| 0.9714 | 15.0 | 4995 | 0.6897 | 0.9069 |
| 0.9141 | 16.0 | 5328 | 0.6327 | 0.9047 |
| 0.8937 | 17.0 | 5661 | 0.5862 | 0.9065 |
| 0.79 | 18.0 | 5994 | 0.5389 | 0.9104 |
| 0.6761 | 19.0 | 6327 | 0.5092 | 0.9075 |
| 0.7064 | 20.0 | 6660 | 0.4760 | 0.9162 |
| 0.7224 | 21.0 | 6993 | 0.4502 | 0.9127 |
| 0.712 | 22.0 | 7326 | 0.4289 | 0.913 |
| 0.6541 | 23.0 | 7659 | 0.4088 | 0.913 |
| 0.6338 | 24.0 | 7992 | 0.3914 | 0.9172 |
| 0.6097 | 25.0 | 8325 | 0.3776 | 0.9182 |
| 0.6369 | 26.0 | 8658 | 0.3676 | 0.9155 |
| 0.6007 | 27.0 | 8991 | 0.3632 | 0.9149 |
| 0.6059 | 28.0 | 9324 | 0.3552 | 0.9187 |
| 0.5227 | 29.0 | 9657 | 0.3454 | 0.9178 |
| 0.6712 | 30.0 | 9990 | 0.3375 | 0.9183 |
| 0.5053 | 31.0 | 10323 | 0.3355 | 0.9171 |
| 0.5432 | 32.0 | 10656 | 0.3328 | 0.917 |
| 0.4617 | 33.0 | 10989 | 0.3295 | 0.9191 |
| 0.4784 | 34.0 | 11322 | 0.3250 | 0.918 |
| 0.5088 | 35.0 | 11655 | 0.3188 | 0.9195 |
| 0.5121 | 36.0 | 11988 | 0.3188 | 0.9172 |
| 0.4734 | 37.0 | 12321 | 0.3174 | 0.9193 |
| 0.5554 | 38.0 | 12654 | 0.3108 | 0.9196 |
| 0.4573 | 39.0 | 12987 | 0.3111 | 0.9203 |
| 0.4692 | 40.0 | 13320 | 0.3074 | 0.9203 |
| 0.481 | 41.0 | 13653 | 0.3042 | 0.922 |
| 0.4888 | 42.0 | 13986 | 0.3058 | 0.921 |
| 0.4032 | 43.0 | 14319 | 0.3025 | 0.9211 |
| 0.4731 | 44.0 | 14652 | 0.3063 | 0.9202 |
| 0.4574 | 45.0 | 14985 | 0.3052 | 0.92 |
| 0.3993 | 46.0 | 15318 | 0.3098 | 0.9215 |
| 0.4631 | 47.0 | 15651 | 0.3078 | 0.9201 |
| 0.409 | 48.0 | 15984 | 0.3056 | 0.9197 |
| 0.4584 | 49.0 | 16317 | 0.3060 | 0.9208 |
| 0.3853 | 50.0 | 16650 | 0.3061 | 0.9208 |
| 0.3836 | 51.0 | 16983 | 0.3072 | 0.9216 |
| 0.3969 | 52.0 | 17316 | 0.3070 | 0.9197 |
| 0.453 | 53.0 | 17649 | 0.3060 | 0.9188 |
| 0.3802 | 54.0 | 17982 | 0.3046 | 0.9204 |
| 0.4191 | 55.0 | 18315 | 0.3075 | 0.9208 |
| 0.4245 | 56.0 | 18648 | 0.3018 | 0.9205 |
| 0.4356 | 57.0 | 18981 | 0.3033 | 0.9214 |
| 0.348 | 58.0 | 19314 | 0.3081 | 0.9208 |
| 0.4232 | 59.0 | 19647 | 0.3058 | 0.9198 |
| 0.3363 | 60.0 | 19980 | 0.3066 | 0.9195 |
| 0.3537 | 61.0 | 20313 | 0.3067 | 0.9197 |
| 0.3613 | 62.0 | 20646 | 0.3065 | 0.9192 |
| 0.4121 | 63.0 | 20979 | 0.3086 | 0.9211 |
| 0.3939 | 64.0 | 21312 | 0.3095 | 0.9207 |
| 0.3616 | 65.0 | 21645 | 0.3061 | 0.9215 |
| 0.3645 | 66.0 | 21978 | 0.3085 | 0.9197 |
| 0.42 | 67.0 | 22311 | 0.3088 | 0.9191 |
| 0.3862 | 68.0 | 22644 | 0.3083 | 0.9193 |
| 0.3519 | 69.0 | 22977 | 0.3103 | 0.9187 |
| 0.4464 | 70.0 | 23310 | 0.3111 | 0.9192 |
| 0.3852 | 71.0 | 23643 | 0.3116 | 0.919 |
| 0.3406 | 72.0 | 23976 | 0.3082 | 0.9194 |
| 0.3785 | 73.0 | 24309 | 0.3071 | 0.9191 |
| 0.3559 | 74.0 | 24642 | 0.3101 | 0.9194 |
| 0.3298 | 75.0 | 24975 | 0.3099 | 0.9187 |
| 0.3596 | 76.0 | 25308 | 0.3099 | 0.9208 |
| 0.3419 | 77.0 | 25641 | 0.3120 | 0.9201 |
| 0.3918 | 78.0 | 25974 | 0.3077 | 0.9201 |
| 0.3571 | 79.0 | 26307 | 0.3119 | 0.9195 |
| 0.3609 | 80.0 | 26640 | 0.3120 | 0.9195 |
| 0.3324 | 81.0 | 26973 | 0.3120 | 0.9194 |
| 0.3387 | 82.0 | 27306 | 0.3118 | 0.9199 |
| 0.441 | 83.0 | 27639 | 0.3117 | 0.92 |
| 0.359 | 84.0 | 27972 | 0.3132 | 0.9195 |
| 0.3106 | 85.0 | 28305 | 0.3131 | 0.9204 |
| 0.3191 | 86.0 | 28638 | 0.3130 | 0.9201 |
| 0.3987 | 87.0 | 28971 | 0.3141 | 0.9202 |
| 0.3327 | 88.0 | 29304 | 0.3138 | 0.9194 |
| 0.3464 | 89.0 | 29637 | 0.3142 | 0.9207 |
| 0.3634 | 90.0 | 29970 | 0.3145 | 0.9207 |
| 0.3123 | 91.0 | 30303 | 0.3133 | 0.9197 |
| 0.3029 | 92.0 | 30636 | 0.3138 | 0.92 |
| 0.3814 | 93.0 | 30969 | 0.3124 | 0.9192 |
| 0.2953 | 94.0 | 31302 | 0.3126 | 0.9203 |
| 0.3475 | 95.0 | 31635 | 0.3141 | 0.9206 |
| 0.3406 | 96.0 | 31968 | 0.3141 | 0.9197 |
| 0.3448 | 97.0 | 32301 | 0.3141 | 0.9198 |
| 0.3687 | 98.0 | 32634 | 0.3137 | 0.9205 |
| 0.345 | 99.0 | 32967 | 0.3144 | 0.92 |
| 0.3582 | 100.0 | 33300 | 0.3142 | 0.9197 |
Framework versions
- Transformers 4.39.3
- Pytorch 2.2.2+cu118
- Datasets 2.18.0
- Tokenizers 0.15.2