distilbert-finetuned-clinc

This model is a fine-tuned version of distilbert/distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.4940
Accuracy: 0.9510
F1: 0.9499
Recall: 0.9510

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 64
eval_batch_size: 64
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Recall
No log	0.4184	100	3.3801	0.7613	0.7344	0.7613
No log	0.8368	200	1.0856	0.8977	0.8960	0.8977
No log	1.2552	300	0.7844	0.9297	0.9293	0.9297
No log	1.6736	400	0.7844	0.9313	0.9308	0.9313
2.3595	2.0921	500	0.7503	0.9374	0.9366	0.9374
2.3595	2.5105	600	0.7324	0.9339	0.9323	0.9339
2.3595	2.9289	700	0.6819	0.9410	0.9398	0.9410
2.3595	3.3473	800	0.6990	0.9371	0.9348	0.9371
2.3595	3.7657	900	0.6468	0.9439	0.9433	0.9439
0.2374	4.1841	1000	0.6520	0.9416	0.9407	0.9416
0.2374	4.6025	1100	0.6681	0.9365	0.9343	0.9365
0.2374	5.0209	1200	0.6545	0.9403	0.9388	0.9403
0.2374	5.4393	1300	0.6016	0.9423	0.9406	0.9423
0.2374	5.8577	1400	0.5688	0.9448	0.9439	0.9448
0.1544	6.2762	1500	0.5705	0.9448	0.9436	0.9448
0.1544	6.6946	1600	0.5655	0.9477	0.9468	0.9477
0.1544	7.1130	1700	0.5683	0.9468	0.9456	0.9468
0.1544	7.5314	1800	0.5392	0.95	0.9490	0.95
0.1544	7.9498	1900	0.5115	0.9497	0.9486	0.9497
0.1101	8.3682	2000	0.5154	0.9497	0.9487	0.9497
0.1101	8.7866	2100	0.5025	0.9510	0.9499	0.9510
0.1101	9.2050	2200	0.4980	0.9506	0.9496	0.9506
0.1101	9.6234	2300	0.4940	0.9510	0.9499	0.9510

Framework versions

Transformers 4.56.1
Pytorch 2.8.0+cu126
Datasets 4.0.0
Tokenizers 0.22.0

Loading…

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002

train_batch_size: 64

eval_batch_size: 64

seed: 42

optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments

lr_scheduler_type: linear

num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Recall
No log	0.4184	100	3.3801	0.7613	0.7344	0.7613
No log	0.8368	200	1.0856	0.8977	0.8960	0.8977
No log	1.2552	300	0.7844	0.9297	0.9293	0.9297
No log	1.6736	400	0.7844	0.9313	0.9308	0.9313
2.3595	2.0921	500	0.7503	0.9374	0.9366	0.9374
2.3595	2.5105	600	0.7324	0.9339	0.9323	0.9339
2.3595	2.9289	700	0.6819	0.9410	0.9398	0.9410
2.3595	3.3473	800	0.6990	0.9371	0.9348	0.9371
2.3595	3.7657	900	0.6468	0.9439	0.9433	0.9439
0.2374	4.1841	1000	0.6520	0.9416	0.9407	0.9416
0.2374	4.6025	1100	0.6681	0.9365	0.9343	0.9365
0.2374	5.0209	1200	0.6545	0.9403	0.9388	0.9403
0.2374	5.4393	1300	0.6016	0.9423	0.9406	0.9423
0.2374	5.8577	1400	0.5688	0.9448	0.9439	0.9448
0.1544	6.2762	1500	0.5705	0.9448	0.9436	0.9448
0.1544	6.6946	1600	0.5655	0.9477	0.9468	0.9477
0.1544	7.1130	1700	0.5683	0.9468	0.9456	0.9468
0.1544	7.5314	1800	0.5392	0.95	0.9490	0.95
0.1544	7.9498	1900	0.5115	0.9497	0.9486	0.9497
0.1101	8.3682	2000	0.5154	0.9497	0.9487	0.9497
0.1101	8.7866	2100	0.5025	0.9510	0.9499	0.9510
0.1101	9.2050	2200	0.4980	0.9506	0.9496	0.9506
0.1101	9.6234	2300	0.4940	0.9510	0.9499	0.9510

Framework versions

Transformers 4.56.1

Pytorch 2.8.0+cu126

Datasets 4.0.0

Tokenizers 0.22.0

pjool/distilbert-finetuned-clinc

distilbert-finetuned-clinc

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

distilbert-finetuned-clinc

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions