Deprem Niyet Sınıflandırma (Dataset v1.3, BERT 128k)
Alakasız sınıfı atılarak eğitildi.
Eval Results
precision recall f1-score support
Lojistik 0.83 0.86 0.84 22
Elektrik Kaynagi 0.71 0.95 0.81 39
Arama Ekipmani 0.72 0.80 0.76 82
Cenaze 0.50 0.33 0.40 3
Giysi 0.79 0.96 0.87 91
Enkaz Kaldirma 0.99 0.95 0.97 601
Isinma 0.75 0.90 0.82 112
Barınma 0.98 0.95 0.96 292
Tuvalet 0.83 1.00 0.91 5
Su 0.80 0.85 0.83 39
Yemek 0.94 0.95 0.94 138
Saglik 0.80 0.85 0.83 75
micro avg 0.90 0.93 0.92 1499
macro avg 0.80 0.86 0.83 1499
weighted avg 0.91 0.93 0.92 1499
samples avg 0.94 0.95 0.94 1499
Reproducibility icin trainer arg'lari:
TrainingArguments(
fp16=True,
evaluation_strategy = "steps",
save_strategy = "steps",
learning_rate=5.1058553791201954e-05,
per_device_train_batch_size=batch_size,
per_device_eval_batch_size=batch_size*2,
num_train_epochs=4,
load_best_model_at_end=True,
metric_for_best_model="macro f1",
logging_steps = step_size,
seed = 42,
data_seed = 42,
dataloader_num_workers = 0,
lr_scheduler_type ="linear",
warmup_steps=0,
weight_decay=0.06437697487126866,
full_determinism = True,
group_by_length = True
)
Threshold: Best Threshold: 0.52