bert-finetuned-ner-assignment2

This model is a fine-tuned version of bert-base-cased on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.2445
Art: {'precision': 0.407, 'recall': 0.599, 'f1': 0.485, 'number': 252.0}
Con: {'precision': 0.343, 'recall': 0.558, 'f1': 0.425, 'number': 163.0}
Loc: {'precision': 0.686, 'recall': 0.732, 'f1': 0.708, 'number': 149.0}
Mat: {'precision': 0.325, 'recall': 0.325, 'f1': 0.325, 'number': 40.0}
Per: {'precision': 0.659, 'recall': 0.816, 'f1': 0.729, 'number': 337.0}
Spe: {'precision': 0.397, 'recall': 0.742, 'f1': 0.517, 'number': 31.0}
Overall Precision: 0.505
Overall Recall: 0.681
Overall F1: 0.58
Overall Accuracy: 0.947
B-art: {'precision': 0.527, 'recall': 0.706, 'f1-score': 0.603, 'support': 252.0}
B-con: {'precision': 0.417, 'recall': 0.65, 'f1-score': 0.508, 'support': 163.0}
B-loc: {'precision': 0.807, 'recall': 0.812, 'f1-score': 0.809, 'support': 149.0}
B-mat: {'precision': 0.457, 'recall': 0.4, 'f1-score': 0.427, 'support': 40.0}
B-per: {'precision': 0.77, 'recall': 0.902, 'f1-score': 0.831, 'support': 337.0}
B-spe: {'precision': 0.483, 'recall': 0.903, 'f1-score': 0.629, 'support': 31.0}
I-art: {'precision': 0.582, 'recall': 0.462, 'f1-score': 0.515, 'support': 353.0}
I-con: {'precision': 0.477, 'recall': 0.438, 'f1-score': 0.457, 'support': 121.0}
I-loc: {'precision': 0.869, 'recall': 0.811, 'f1-score': 0.839, 'support': 254.0}
I-mat: {'precision': 0.5, 'recall': 0.135, 'f1-score': 0.213, 'support': 37.0}
I-per: {'precision': 0.851, 'recall': 0.754, 'f1-score': 0.8, 'support': 447.0}
I-spe: {'precision': 0.838, 'recall': 0.738, 'f1-score': 0.785, 'support': 42.0}
O: {'precision': 0.977, 'recall': 0.974, 'f1-score': 0.975, 'support': 20705.0}
Accuracy: 0.947
Macro avg: {'precision': 0.658, 'recall': 0.668, 'f1-score': 0.645, 'support': 22931.0}
Weighted avg: {'precision': 0.949, 'recall': 0.947, 'f1-score': 0.947, 'support': 22931.0}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Art	Con	Loc	Mat	Per	Spe	Overall Precision	Overall Recall	Overall F1	Overall Accuracy	B-art	B-con	B-loc	B-mat	B-per	B-spe	I-art	I-con	I-loc	I-mat	I-per	I-spe	O	Accuracy	Macro avg	Weighted avg
No log	1.0	249	0.2132	{'precision': 0.403, 'recall': 0.571, 'f1': 0.473, 'number': 252.0}	{'precision': 0.345, 'recall': 0.54, 'f1': 0.421, 'number': 163.0}	{'precision': 0.597, 'recall': 0.638, 'f1': 0.617, 'number': 149.0}	{'precision': 0.368, 'recall': 0.175, 'f1': 0.237, 'number': 40.0}	{'precision': 0.63, 'recall': 0.727, 'f1': 0.675, 'number': 337.0}	{'precision': 0.377, 'recall': 0.742, 'f1': 0.5, 'number': 31.0}	0.485	0.619	0.544	0.945	{'precision': 0.538, 'recall': 0.675, 'f1-score': 0.599, 'support': 252.0}	{'precision': 0.424, 'recall': 0.62, 'f1-score': 0.504, 'support': 163.0}	{'precision': 0.763, 'recall': 0.779, 'f1-score': 0.771, 'support': 149.0}	{'precision': 0.526, 'recall': 0.25, 'f1-score': 0.339, 'support': 40.0}	{'precision': 0.763, 'recall': 0.861, 'f1-score': 0.809, 'support': 337.0}	{'precision': 0.467, 'recall': 0.903, 'f1-score': 0.615, 'support': 31.0}	{'precision': 0.579, 'recall': 0.428, 'f1-score': 0.492, 'support': 353.0}	{'precision': 0.467, 'recall': 0.405, 'f1-score': 0.434, 'support': 121.0}	{'precision': 0.902, 'recall': 0.724, 'f1-score': 0.803, 'support': 254.0}	{'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}	{'precision': 0.93, 'recall': 0.651, 'f1-score': 0.766, 'support': 447.0}	{'precision': 0.861, 'recall': 0.738, 'f1-score': 0.795, 'support': 42.0}	{'precision': 0.971, 'recall': 0.978, 'f1-score': 0.975, 'support': 20705.0}	0.945	{'precision': 0.63, 'recall': 0.616, 'f1-score': 0.608, 'support': 22931.0}	{'precision': 0.945, 'recall': 0.945, 'f1-score': 0.944, 'support': 22931.0}
No log	2.0	498	0.2182	{'precision': 0.419, 'recall': 0.567, 'f1': 0.482, 'number': 252.0}	{'precision': 0.453, 'recall': 0.497, 'f1': 0.474, 'number': 163.0}	{'precision': 0.669, 'recall': 0.678, 'f1': 0.673, 'number': 149.0}	{'precision': 0.333, 'recall': 0.475, 'f1': 0.392, 'number': 40.0}	{'precision': 0.675, 'recall': 0.777, 'f1': 0.723, 'number': 337.0}	{'precision': 0.346, 'recall': 0.581, 'f1': 0.434, 'number': 31.0}	0.534	0.642	0.583	0.95	{'precision': 0.535, 'recall': 0.675, 'f1-score': 0.596, 'support': 252.0}	{'precision': 0.526, 'recall': 0.564, 'f1-score': 0.544, 'support': 163.0}	{'precision': 0.817, 'recall': 0.779, 'f1-score': 0.797, 'support': 149.0}	{'precision': 0.434, 'recall': 0.575, 'f1-score': 0.495, 'support': 40.0}	{'precision': 0.782, 'recall': 0.864, 'f1-score': 0.821, 'support': 337.0}	{'precision': 0.49, 'recall': 0.774, 'f1-score': 0.6, 'support': 31.0}	{'precision': 0.643, 'recall': 0.414, 'f1-score': 0.503, 'support': 353.0}	{'precision': 0.585, 'recall': 0.397, 'f1-score': 0.473, 'support': 121.0}	{'precision': 0.901, 'recall': 0.791, 'f1-score': 0.843, 'support': 254.0}	{'precision': 0.533, 'recall': 0.216, 'f1-score': 0.308, 'support': 37.0}	{'precision': 0.887, 'recall': 0.756, 'f1-score': 0.816, 'support': 447.0}	{'precision': 0.903, 'recall': 0.667, 'f1-score': 0.767, 'support': 42.0}	{'precision': 0.973, 'recall': 0.98, 'f1-score': 0.977, 'support': 20705.0}	0.95	{'precision': 0.693, 'recall': 0.65, 'f1-score': 0.657, 'support': 22931.0}	{'precision': 0.949, 'recall': 0.95, 'f1-score': 0.949, 'support': 22931.0}
0.1430	3.0	747	0.2445	{'precision': 0.407, 'recall': 0.599, 'f1': 0.485, 'number': 252.0}	{'precision': 0.343, 'recall': 0.558, 'f1': 0.425, 'number': 163.0}	{'precision': 0.686, 'recall': 0.732, 'f1': 0.708, 'number': 149.0}	{'precision': 0.325, 'recall': 0.325, 'f1': 0.325, 'number': 40.0}	{'precision': 0.659, 'recall': 0.816, 'f1': 0.729, 'number': 337.0}	{'precision': 0.397, 'recall': 0.742, 'f1': 0.517, 'number': 31.0}	0.505	0.681	0.58	0.947	{'precision': 0.527, 'recall': 0.706, 'f1-score': 0.603, 'support': 252.0}	{'precision': 0.417, 'recall': 0.65, 'f1-score': 0.508, 'support': 163.0}	{'precision': 0.807, 'recall': 0.812, 'f1-score': 0.809, 'support': 149.0}	{'precision': 0.457, 'recall': 0.4, 'f1-score': 0.427, 'support': 40.0}	{'precision': 0.77, 'recall': 0.902, 'f1-score': 0.831, 'support': 337.0}	{'precision': 0.483, 'recall': 0.903, 'f1-score': 0.629, 'support': 31.0}	{'precision': 0.582, 'recall': 0.462, 'f1-score': 0.515, 'support': 353.0}	{'precision': 0.477, 'recall': 0.438, 'f1-score': 0.457, 'support': 121.0}	{'precision': 0.869, 'recall': 0.811, 'f1-score': 0.839, 'support': 254.0}	{'precision': 0.5, 'recall': 0.135, 'f1-score': 0.213, 'support': 37.0}	{'precision': 0.851, 'recall': 0.754, 'f1-score': 0.8, 'support': 447.0}	{'precision': 0.838, 'recall': 0.738, 'f1-score': 0.785, 'support': 42.0}	{'precision': 0.977, 'recall': 0.974, 'f1-score': 0.975, 'support': 20705.0}	0.947	{'precision': 0.658, 'recall': 0.668, 'f1-score': 0.645, 'support': 22931.0}	{'precision': 0.949, 'recall': 0.947, 'f1-score': 0.947, 'support': 22931.0}

Framework versions

Transformers 5.22.1
Pytorch 2.14.1+cpu
Datasets 4.8.5
Tokenizers 0.23.3

Loading…

bert-finetuned-ner-assignment2

This model is a fine-tuned version of bert-base-cased on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.2445

Art: {'precision': 0.407, 'recall': 0.599, 'f1': 0.485, 'number': 252.0}

Con: {'precision': 0.343, 'recall': 0.558, 'f1': 0.425, 'number': 163.0}

Loc: {'precision': 0.686, 'recall': 0.732, 'f1': 0.708, 'number': 149.0}

Mat: {'precision': 0.325, 'recall': 0.325, 'f1': 0.325, 'number': 40.0}

Per: {'precision': 0.659, 'recall': 0.816, 'f1': 0.729, 'number': 337.0}

Spe: {'precision': 0.397, 'recall': 0.742, 'f1': 0.517, 'number': 31.0}

Overall Precision: 0.505

Overall Recall: 0.681

Overall F1: 0.58

Overall Accuracy: 0.947

B-art: {'precision': 0.527, 'recall': 0.706, 'f1-score': 0.603, 'support': 252.0}

B-con: {'precision': 0.417, 'recall': 0.65, 'f1-score': 0.508, 'support': 163.0}

B-loc: {'precision': 0.807, 'recall': 0.812, 'f1-score': 0.809, 'support': 149.0}

B-mat: {'precision': 0.457, 'recall': 0.4, 'f1-score': 0.427, 'support': 40.0}

B-per: {'precision': 0.77, 'recall': 0.902, 'f1-score': 0.831, 'support': 337.0}

B-spe: {'precision': 0.483, 'recall': 0.903, 'f1-score': 0.629, 'support': 31.0}

I-art: {'precision': 0.582, 'recall': 0.462, 'f1-score': 0.515, 'support': 353.0}

I-con: {'precision': 0.477, 'recall': 0.438, 'f1-score': 0.457, 'support': 121.0}

I-loc: {'precision': 0.869, 'recall': 0.811, 'f1-score': 0.839, 'support': 254.0}

I-mat: {'precision': 0.5, 'recall': 0.135, 'f1-score': 0.213, 'support': 37.0}

I-per: {'precision': 0.851, 'recall': 0.754, 'f1-score': 0.8, 'support': 447.0}

I-spe: {'precision': 0.838, 'recall': 0.738, 'f1-score': 0.785, 'support': 42.0}

O: {'precision': 0.977, 'recall': 0.974, 'f1-score': 0.975, 'support': 20705.0}

Accuracy: 0.947

Macro avg: {'precision': 0.658, 'recall': 0.668, 'f1-score': 0.645, 'support': 22931.0}

Weighted avg: {'precision': 0.949, 'recall': 0.947, 'f1-score': 0.947, 'support': 22931.0}

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05

train_batch_size: 8

eval_batch_size: 8

seed: 42

optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments

lr_scheduler_type: linear

num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Art	Con	Loc	Mat	Per	Spe	Overall Precision	Overall Recall	Overall F1	Overall Accuracy	B-art	B-con	B-loc	B-mat	B-per	B-spe	I-art	I-con	I-loc	I-mat	I-per	I-spe	O	Accuracy	Macro avg	Weighted avg
No log	1.0	249	0.2132	{'precision': 0.403, 'recall': 0.571, 'f1': 0.473, 'number': 252.0}	{'precision': 0.345, 'recall': 0.54, 'f1': 0.421, 'number': 163.0}	{'precision': 0.597, 'recall': 0.638, 'f1': 0.617, 'number': 149.0}	{'precision': 0.368, 'recall': 0.175, 'f1': 0.237, 'number': 40.0}	{'precision': 0.63, 'recall': 0.727, 'f1': 0.675, 'number': 337.0}	{'precision': 0.377, 'recall': 0.742, 'f1': 0.5, 'number': 31.0}	0.485	0.619	0.544	0.945	{'precision': 0.538, 'recall': 0.675, 'f1-score': 0.599, 'support': 252.0}	{'precision': 0.424, 'recall': 0.62, 'f1-score': 0.504, 'support': 163.0}	{'precision': 0.763, 'recall': 0.779, 'f1-score': 0.771, 'support': 149.0}	{'precision': 0.526, 'recall': 0.25, 'f1-score': 0.339, 'support': 40.0}	{'precision': 0.763, 'recall': 0.861, 'f1-score': 0.809, 'support': 337.0}	{'precision': 0.467, 'recall': 0.903, 'f1-score': 0.615, 'support': 31.0}	{'precision': 0.579, 'recall': 0.428, 'f1-score': 0.492, 'support': 353.0}	{'precision': 0.467, 'recall': 0.405, 'f1-score': 0.434, 'support': 121.0}	{'precision': 0.902, 'recall': 0.724, 'f1-score': 0.803, 'support': 254.0}	{'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 37.0}	{'precision': 0.93, 'recall': 0.651, 'f1-score': 0.766, 'support': 447.0}	{'precision': 0.861, 'recall': 0.738, 'f1-score': 0.795, 'support': 42.0}	{'precision': 0.971, 'recall': 0.978, 'f1-score': 0.975, 'support': 20705.0}	0.945	{'precision': 0.63, 'recall': 0.616, 'f1-score': 0.608, 'support': 22931.0}	{'precision': 0.945, 'recall': 0.945, 'f1-score': 0.944, 'support': 22931.0}
No log	2.0	498	0.2182	{'precision': 0.419, 'recall': 0.567, 'f1': 0.482, 'number': 252.0}	{'precision': 0.453, 'recall': 0.497, 'f1': 0.474, 'number': 163.0}	{'precision': 0.669, 'recall': 0.678, 'f1': 0.673, 'number': 149.0}	{'precision': 0.333, 'recall': 0.475, 'f1': 0.392, 'number': 40.0}	{'precision': 0.675, 'recall': 0.777, 'f1': 0.723, 'number': 337.0}	{'precision': 0.346, 'recall': 0.581, 'f1': 0.434, 'number': 31.0}	0.534	0.642	0.583	0.95	{'precision': 0.535, 'recall': 0.675, 'f1-score': 0.596, 'support': 252.0}	{'precision': 0.526, 'recall': 0.564, 'f1-score': 0.544, 'support': 163.0}	{'precision': 0.817, 'recall': 0.779, 'f1-score': 0.797, 'support': 149.0}	{'precision': 0.434, 'recall': 0.575, 'f1-score': 0.495, 'support': 40.0}	{'precision': 0.782, 'recall': 0.864, 'f1-score': 0.821, 'support': 337.0}	{'precision': 0.49, 'recall': 0.774, 'f1-score': 0.6, 'support': 31.0}	{'precision': 0.643, 'recall': 0.414, 'f1-score': 0.503, 'support': 353.0}	{'precision': 0.585, 'recall': 0.397, 'f1-score': 0.473, 'support': 121.0}	{'precision': 0.901, 'recall': 0.791, 'f1-score': 0.843, 'support': 254.0}	{'precision': 0.533, 'recall': 0.216, 'f1-score': 0.308, 'support': 37.0}	{'precision': 0.887, 'recall': 0.756, 'f1-score': 0.816, 'support': 447.0}	{'precision': 0.903, 'recall': 0.667, 'f1-score': 0.767, 'support': 42.0}	{'precision': 0.973, 'recall': 0.98, 'f1-score': 0.977, 'support': 20705.0}	0.95	{'precision': 0.693, 'recall': 0.65, 'f1-score': 0.657, 'support': 22931.0}	{'precision': 0.949, 'recall': 0.95, 'f1-score': 0.949, 'support': 22931.0}
0.1430	3.0	747	0.2445	{'precision': 0.407, 'recall': 0.599, 'f1': 0.485, 'number': 252.0}	{'precision': 0.343, 'recall': 0.558, 'f1': 0.425, 'number': 163.0}	{'precision': 0.686, 'recall': 0.732, 'f1': 0.708, 'number': 149.0}	{'precision': 0.325, 'recall': 0.325, 'f1': 0.325, 'number': 40.0}	{'precision': 0.659, 'recall': 0.816, 'f1': 0.729, 'number': 337.0}	{'precision': 0.397, 'recall': 0.742, 'f1': 0.517, 'number': 31.0}	0.505	0.681	0.58	0.947	{'precision': 0.527, 'recall': 0.706, 'f1-score': 0.603, 'support': 252.0}	{'precision': 0.417, 'recall': 0.65, 'f1-score': 0.508, 'support': 163.0}	{'precision': 0.807, 'recall': 0.812, 'f1-score': 0.809, 'support': 149.0}	{'precision': 0.457, 'recall': 0.4, 'f1-score': 0.427, 'support': 40.0}	{'precision': 0.77, 'recall': 0.902, 'f1-score': 0.831, 'support': 337.0}	{'precision': 0.483, 'recall': 0.903, 'f1-score': 0.629, 'support': 31.0}	{'precision': 0.582, 'recall': 0.462, 'f1-score': 0.515, 'support': 353.0}	{'precision': 0.477, 'recall': 0.438, 'f1-score': 0.457, 'support': 121.0}	{'precision': 0.869, 'recall': 0.811, 'f1-score': 0.839, 'support': 254.0}	{'precision': 0.5, 'recall': 0.135, 'f1-score': 0.213, 'support': 37.0}	{'precision': 0.851, 'recall': 0.754, 'f1-score': 0.8, 'support': 447.0}	{'precision': 0.838, 'recall': 0.738, 'f1-score': 0.785, 'support': 42.0}	{'precision': 0.977, 'recall': 0.974, 'f1-score': 0.975, 'support': 20705.0}	0.947	{'precision': 0.658, 'recall': 0.668, 'f1-score': 0.645, 'support': 22931.0}	{'precision': 0.949, 'recall': 0.947, 'f1-score': 0.947, 'support': 22931.0}

Framework versions

Transformers 5.22.1

Pytorch 2.14.1+cpu

Datasets 4.8.5

Tokenizers 0.23.3

brittnismith/bert-finetuned-ner-assignment2

bert-finetuned-ner-assignment2

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

bert-finetuned-ner-assignment2

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions