roberta-medium-amharic-32k-256-512
This model is a fine-tuned version of yosefw/roberta-medium-amharic-32k-256 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 2.4505
- Model Preparation Time: 0.0027
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- lr_scheduler_warmup_steps: 1000
- num_epochs: 4
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Model Preparation Time |
|---|---|---|---|---|
| 6.0763 | 0.1250 | 1097 | 5.6672 | 0.0027 |
| 3.9591 | 0.2500 | 2194 | 2.6870 | 0.0027 |
| 2.8872 | 0.3750 | 3291 | 2.5964 | 0.0027 |
| 2.793 | 0.4999 | 4388 | 2.5810 | 0.0027 |
| 2.75 | 0.6249 | 5485 | 2.5584 | 0.0027 |
| 2.7304 | 0.7499 | 6582 | 2.5505 | 0.0027 |
| 2.7112 | 0.8749 | 7679 | 2.5273 | 0.0027 |
| 2.7022 | 0.9999 | 8776 | 2.5210 | 0.0027 |
| 2.6867 | 1.1249 | 9873 | 2.5225 | 0.0027 |
| 2.6741 | 1.2499 | 10970 | 2.5016 | 0.0027 |
| 2.6583 | 1.3748 | 12067 | 2.4972 | 0.0027 |
| 2.6558 | 1.4998 | 13164 | 2.4859 | 0.0027 |
| 2.6483 | 1.6248 | 14261 | 2.4869 | 0.0027 |
| 2.6454 | 1.7498 | 15358 | 2.4878 | 0.0027 |
| 2.6366 | 1.8748 | 16455 | 2.4719 | 0.0027 |
| 2.6395 | 1.9998 | 17552 | 2.4837 | 0.0027 |
| 2.6338 | 2.1248 | 18649 | 2.4687 | 0.0027 |
| 2.625 | 2.2497 | 19746 | 2.4732 | 0.0027 |
| 2.6248 | 2.3747 | 20843 | 2.4731 | 0.0027 |
| 2.6202 | 2.4997 | 21940 | 2.4654 | 0.0027 |
| 2.6177 | 2.6247 | 23037 | 2.4574 | 0.0027 |
| 2.6189 | 2.7497 | 24134 | 2.4712 | 0.0027 |
| 2.6142 | 2.8747 | 25231 | 2.4683 | 0.0027 |
| 2.6108 | 2.9997 | 26328 | 2.4574 | 0.0027 |
| 2.6086 | 3.1246 | 27425 | 2.4553 | 0.0027 |
| 2.6086 | 3.2496 | 28522 | 2.4488 | 0.0027 |
| 2.6074 | 3.3746 | 29619 | 2.4628 | 0.0027 |
| 2.6056 | 3.4996 | 30716 | 2.4583 | 0.0027 |
| 2.6026 | 3.6246 | 31813 | 2.4519 | 0.0027 |
| 2.5996 | 3.7496 | 32910 | 2.4523 | 0.0027 |
| 2.6083 | 3.8746 | 34007 | 2.4489 | 0.0027 |
| 2.6007 | 3.9995 | 35104 | 2.4462 | 0.0027 |
Framework versions
- Transformers 4.47.1
- Pytorch 2.5.1+cu121
- Datasets 3.2.0
- Tokenizers 0.21.0