bertbek-ner-uznews
This model is a fine-tuned version of elmurod1202/bertbek-news-big-cased on UzNER dataset. It achieves the following results on the evaluation set:
- Loss: 0.1255
- Date: {'precision': 0.8585858585858586, 'recall': 0.9205776173285198, 'f1': 0.8885017421602788, 'number': 277}
- Location: {'precision': 0.7639344262295082, 'recall': 0.7871621621621622, 'f1': 0.7753743760399334, 'number': 296}
- Misc: {'precision': 0.5661375661375662, 'recall': 0.622093023255814, 'f1': 0.5927977839335179, 'number': 172}
- Org: {'precision': 0.5685279187817259, 'recall': 0.5743589743589743, 'f1': 0.5714285714285714, 'number': 195}
- Person: {'precision': 0.8644501278772379, 'recall': 0.9415041782729805, 'f1': 0.9013333333333333, 'number': 359}
- Time: {'precision': 0.14285714285714285, 'recall': 0.1, 'f1': 0.11764705882352941, 'number': 10}
- Overall Precision: 0.7547
- Overall Recall: 0.7991
- Overall F1: 0.7763
- Overall Accuracy: 0.9616
Model description
BERTbek model fine-tuned for the NER task
Training and evaluation data
UzNER dataset
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 4
Training results
| Training Loss | Epoch | Step | Validation Loss | Date | Location | Misc | Org | Person | Time | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 159 | 0.2034 | {'precision': 0.7120743034055728, 'recall': 0.8303249097472925, 'f1': 0.7666666666666667, 'number': 277} | {'precision': 0.7021943573667712, 'recall': 0.7567567567567568, 'f1': 0.7284552845528456, 'number': 296} | {'precision': 0.43902439024390244, 'recall': 0.20930232558139536, 'f1': 0.28346456692913385, 'number': 172} | {'precision': 0.4134078212290503, 'recall': 0.37948717948717947, 'f1': 0.3957219251336898, 'number': 195} | {'precision': 0.6337349397590362, 'recall': 0.7325905292479109, 'f1': 0.6795865633074936, 'number': 359} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 10} | 0.6256 | 0.6318 | 0.6287 | 0.9397 |
| No log | 2.0 | 318 | 0.1454 | {'precision': 0.839041095890411, 'recall': 0.8844765342960289, 'f1': 0.8611599297012302, 'number': 277} | {'precision': 0.7539936102236422, 'recall': 0.7972972972972973, 'f1': 0.7750410509031198, 'number': 296} | {'precision': 0.5465116279069767, 'recall': 0.5465116279069767, 'f1': 0.5465116279069767, 'number': 172} | {'precision': 0.5721649484536082, 'recall': 0.5692307692307692, 'f1': 0.570694087403599, 'number': 195} | {'precision': 0.7985257985257985, 'recall': 0.9052924791086351, 'f1': 0.8485639686684072, 'number': 359} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 10} | 0.7305 | 0.7723 | 0.7508 | 0.9586 |
| No log | 3.0 | 477 | 0.1333 | {'precision': 0.8246753246753247, 'recall': 0.9169675090252708, 'f1': 0.8683760683760684, 'number': 277} | {'precision': 0.7005988023952096, 'recall': 0.7905405405405406, 'f1': 0.7428571428571429, 'number': 296} | {'precision': 0.5204081632653061, 'recall': 0.5930232558139535, 'f1': 0.5543478260869567, 'number': 172} | {'precision': 0.5784313725490197, 'recall': 0.6051282051282051, 'f1': 0.5914786967418547, 'number': 195} | {'precision': 0.850253807106599, 'recall': 0.9331476323119777, 'f1': 0.8897742363877823, 'number': 359} | {'precision': 0.14285714285714285, 'recall': 0.1, 'f1': 0.11764705882352941, 'number': 10} | 0.7235 | 0.7976 | 0.7587 | 0.9583 |
| 0.2001 | 4.0 | 636 | 0.1255 | {'precision': 0.8585858585858586, 'recall': 0.9205776173285198, 'f1': 0.8885017421602788, 'number': 277} | {'precision': 0.7639344262295082, 'recall': 0.7871621621621622, 'f1': 0.7753743760399334, 'number': 296} | {'precision': 0.5661375661375662, 'recall': 0.622093023255814, 'f1': 0.5927977839335179, 'number': 172} | {'precision': 0.5685279187817259, 'recall': 0.5743589743589743, 'f1': 0.5714285714285714, 'number': 195} | {'precision': 0.8644501278772379, 'recall': 0.9415041782729805, 'f1': 0.9013333333333333, 'number': 359} | {'precision': 0.14285714285714285, 'recall': 0.1, 'f1': 0.11764705882352941, 'number': 10} | 0.7547 | 0.7991 | 0.7763 | 0.9616 |
Framework versions
- Transformers 4.50.3
- Pytorch 2.6.0+cu124
- Datasets 3.5.0
- Tokenizers 0.21.1