Query Quality Estimation - Binary Classification
This model is a fine-tuned version of answerdotai/ModernBERT-base on the ymoslem/TeleQnA-router dataset. It achieves the following results on the evaluation set:
- Loss: 3.1483
- Accuracy: 0.698
- F1 Macro: 0.6348
- F1 Weighted: 0.6855
- Precision: 0.6527
- Recall: 0.6293
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 10
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 Macro | F1 Weighted | Precision | Recall |
|---|---|---|---|---|---|---|---|---|
| 0.3158 | 1.0 | 704 | 1.1034 | 0.689 | 0.5779 | 0.6502 | 0.6409 | 0.5827 |
| 0.1318 | 2.0 | 1408 | 1.0367 | 0.64 | 0.6227 | 0.6497 | 0.6254 | 0.6399 |
| 0.0745 | 3.0 | 2112 | 2.0954 | 0.706 | 0.6170 | 0.6787 | 0.6671 | 0.6134 |
| 0.0351 | 4.0 | 2816 | 2.1061 | 0.685 | 0.6195 | 0.6722 | 0.6355 | 0.6150 |
| 0.0335 | 5.0 | 3520 | 2.5072 | 0.697 | 0.6436 | 0.6897 | 0.6532 | 0.6390 |
| 0.0009 | 6.0 | 4224 | 2.3591 | 0.68 | 0.6455 | 0.6824 | 0.6437 | 0.6481 |
| 0.0 | 7.0 | 4928 | 2.7481 | 0.691 | 0.6417 | 0.6861 | 0.6474 | 0.6383 |
| 0.0001 | 8.0 | 5632 | 2.9967 | 0.696 | 0.6338 | 0.6842 | 0.6502 | 0.6285 |
| 0.0 | 9.0 | 6336 | 3.4420 | 0.712 | 0.6321 | 0.6894 | 0.6746 | 0.6262 |
| 0.0 | 10.0 | 7040 | 3.1483 | 0.698 | 0.6348 | 0.6855 | 0.6527 | 0.6293 |
Framework versions
- Transformers 4.57.3
- Pytorch 2.8.0+cu128
- Datasets 4.1.1
- Tokenizers 0.22.1