Training procedure
Generalization capabilities based on this notebook are still lacking.
Test Metrics:
'eval_loss': 0.4365999102592468,
'eval_accuracy': 0.9420512498988455,
'eval_precision': 0.2294216317399737,
'eval_recall': 0.7584797572261781,
'eval_f1': 0.35228549223252686,
'eval_auc': 0.8522129916967822,
'eval_mcc': 0.39725936154292724,
The metrics on the train and test datasets from the paper mentioned in this notebook are:
Train metrics:
Average Accuracy: 0.9169
Average Precision: 0.1416
Average Recall: 0.2769
Average F1 Score: 0.1743
Average AUC: 0.6088
Average MCC: 0.1522
Test metrics:
Average Accuracy: 0.9199
Average Precision: 0.1307
Average Recall: 0.2750
Average F1 Score: 0.1677
Average AUC: 0.6081
Average MCC: 0.1474
Framework versions
- PEFT 0.5.0