careerbot_PG6_google_flan-t5-small_model
This model is a fine-tuned version of google/flan-t5-small on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.6685
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1
- training_steps: 3372
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| No log | 1.0 | 105 | 21.1784 |
| No log | 2.0 | 211 | 7.1162 |
| No log | 3.0 | 316 | 4.7947 |
| No log | 4.0 | 422 | 4.4165 |
| No log | 5.0 | 527 | 4.1394 |
| No log | 6.0 | 633 | 3.7892 |
| No log | 7.0 | 738 | 3.5130 |
| No log | 8.0 | 844 | 3.2547 |
| No log | 9.0 | 949 | 3.0385 |
| No log | 10.0 | 1055 | 2.8504 |
| No log | 11.0 | 1160 | 2.6901 |
| No log | 12.0 | 1266 | 2.5468 |
| No log | 13.0 | 1371 | 2.4245 |
| No log | 14.0 | 1477 | 2.3141 |
| No log | 15.0 | 1582 | 2.2212 |
| No log | 16.0 | 1688 | 2.1392 |
| No log | 17.0 | 1793 | 2.0683 |
| No log | 18.0 | 1899 | 2.0034 |
| No log | 19.0 | 2004 | 1.9474 |
| No log | 20.0 | 2110 | 1.9004 |
| No log | 21.0 | 2215 | 1.8585 |
| No log | 22.0 | 2321 | 1.8215 |
| No log | 23.0 | 2426 | 1.7892 |
| No log | 24.0 | 2532 | 1.7609 |
| No log | 25.0 | 2637 | 1.7387 |
| No log | 26.0 | 2743 | 1.7178 |
| No log | 27.0 | 2848 | 1.7033 |
| No log | 28.0 | 2954 | 1.6899 |
| No log | 29.0 | 3059 | 1.6804 |
| No log | 30.0 | 3165 | 1.6734 |
| No log | 31.0 | 3270 | 1.6694 |
| No log | 31.96 | 3372 | 1.6685 |
Framework versions
- Transformers 4.35.0
- Pytorch 2.5.0+cu124
- Datasets 2.19.0
- Tokenizers 0.14.1