bert-finetuned-pos
This model is a fine-tuned version of bert-base-cased on the conll2003 dataset.
Model description
"This model is used for POS token classification and utilizes the CoNLL-2003 dataset. In processing this model, we used dynamic padding.
Intended uses & limitations
This model can be used for token classification in the field of pos. POS tags" or Part-of-Speech tags are labels that indicate the grammatical role of each word in a sentence (such as noun, verb, adjective, etc.). In a token classification task, each word (token) is individually analyzed to assign the appropriate POS tag. This helps the model better understand sentence structure and improves performance in tasks like translation, summarization, and information extraction
Training and evaluation data
conll2003
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3
Training results
global_step=5268, training_loss=0.08724395379752214, metrics={'train_runtime': 566.4878, 'train_samples_per_second': 74.358, 'train_steps_per_second': 9.299, 'total_flos': 921087900564942.0, 'train_loss': 0.08724395379752214, 'epoch': 3.0}
Framework versions
- Transformers 4.51.3
- Pytorch 2.6.0+cu124
- Datasets 3.5.1
- Tokenizers 0.21.1