Khmer Orthographic Correction System using NLLB
This model is a fine-tuned version of facebook/mbart-large-50 on the khmer-orthography-correction-dataset dataset. It achieves the following results on the evaluation set:
- Loss: 0.2433
- Cer: 0.0612
- Wer: 0.4716
- Bleu: {'score': 54.58228361171526, 'counts': [4449, 517, 149, 31], 'totals': [7551, 819, 273, 65], 'precisions': [58.919348430671434, 63.12576312576313, 54.57875457875458, 47.69230769230769], 'bp': 0.9785151810825774, 'sys_len': 7551, 'ref_len': 7715}
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Cer | Wer | Bleu |
|---|---|---|---|---|---|---|
| No log | 1.0 | 421 | 0.5867 | 0.1395 | 0.7613 | {'score': 31.163336304191414, 'counts': [2533, 357, 88, 18], 'totals': [7662, 930, 292, 71], 'precisions': [33.05925345862699, 38.38709677419355, 30.136986301369863, 25.35211267605634], 'bp': 0.9931066151526365, 'sys_len': 7662, 'ref_len': 7715} |
| 0.8766 | 2.0 | 842 | 0.4476 | 0.1080 | 0.6706 | {'score': 35.87207143179994, 'counts': [3104, 395, 99, 18], 'totals': [7600, 868, 281, 67], 'precisions': [40.8421052631579, 45.50691244239631, 35.23131672597865, 26.865671641791046], 'bp': 0.9849823281382666, 'sys_len': 7600, 'ref_len': 7715} |
| 0.5573 | 3.0 | 1263 | 0.3727 | 0.0906 | 0.6090 | {'score': 42.601695587857506, 'counts': [3510, 436, 115, 22], 'totals': [7572, 840, 272, 63], 'precisions': [46.35499207606973, 51.904761904761905, 42.279411764705884, 34.92063492063492], 'bp': 0.9812918440841442, 'sys_len': 7572, 'ref_len': 7715} |
| 0.4428 | 4.0 | 1684 | 0.3265 | 0.0819 | 0.5647 | {'score': 47.75763819840498, 'counts': [3809, 467, 130, 26], 'totals': [7560, 828, 270, 63], 'precisions': [50.383597883597886, 56.40096618357488, 48.148148148148145, 41.26984126984127], 'bp': 0.9797061046559977, 'sys_len': 7560, 'ref_len': 7715} |
| 0.3718 | 5.0 | 2105 | 0.2955 | 0.0732 | 0.5332 | {'score': 50.28390617740271, 'counts': [4028, 486, 136, 28], 'totals': [7556, 824, 269, 64], 'precisions': [53.30862890418211, 58.980582524271846, 50.55762081784387, 43.75], 'bp': 0.9791769767263626, 'sys_len': 7556, 'ref_len': 7715} |
| 0.3321 | 6.0 | 2526 | 0.2745 | 0.0681 | 0.5079 | {'score': 53.18765542932397, 'counts': [4198, 499, 143, 31], 'totals': [7550, 818, 269, 64], 'precisions': [55.602649006622514, 61.002444987775064, 53.15985130111524, 48.4375], 'bp': 0.9783827705016936, 'sys_len': 7550, 'ref_len': 7715} |
| 0.3321 | 7.0 | 2947 | 0.2600 | 0.0653 | 0.4901 | {'score': 53.42672975882541, 'counts': [4320, 505, 145, 31], 'totals': [7548, 816, 271, 66], 'precisions': [57.233704292527825, 61.88725490196079, 53.505535055350556, 46.96969696969697], 'bp': 0.9781178978708265, 'sys_len': 7548, 'ref_len': 7715} |
| 0.3023 | 8.0 | 3368 | 0.2503 | 0.0636 | 0.4763 | {'score': 54.40154581493312, 'counts': [4417, 512, 149, 31], 'totals': [7551, 819, 272, 65], 'precisions': [58.49556350152298, 62.51526251526251, 54.779411764705884, 47.69230769230769], 'bp': 0.9785151810825774, 'sys_len': 7551, 'ref_len': 7715} |
| 0.2841 | 9.0 | 3789 | 0.2452 | 0.0621 | 0.4727 | {'score': 54.86859193873856, 'counts': [4439, 514, 149, 31], 'totals': [7548, 816, 270, 64], 'precisions': [58.810280869104396, 62.990196078431374, 55.18518518518518, 48.4375], 'bp': 0.9781178978708265, 'sys_len': 7548, 'ref_len': 7715} |
| 0.2744 | 10.0 | 4210 | 0.2433 | 0.0612 | 0.4716 | {'score': 54.58228361171526, 'counts': [4449, 517, 149, 31], 'totals': [7551, 819, 273, 65], 'precisions': [58.919348430671434, 63.12576312576313, 54.57875457875458, 47.69230769230769], 'bp': 0.9785151810825774, 'sys_len': 7551, 'ref_len': 7715} |
Framework versions
- Transformers 4.57.2
- Pytorch 2.9.0+cu126
- Datasets 4.0.0
- Tokenizers 0.22.1