RoBERTa-large fine-tuned on OntoNotes 5.0

This model is a fine-tuned version of FacebookAI/roberta-large on the English subset of the OntoNotes 5.0 (CoNLL-2012) dataset. RoBERTa-large features 24 layers and ~355M parameters, providing enhanced semantic understanding for complex Named Entity Recognition (NER) tasks compared to the base architecture.

📊 Performance

The following results were achieved on the OntoNotes 5.0 (v12) test set:

🛠 Training Details

To optimize the 24-layer transformer on 2xNVIDIA V100 GPUs:

• Architecture: RobertaForTokenClassification
• Tokenizer: RobertaTokenizerFast (with add_prefix_space=True)
• Learning Rate: 1e-5
• Effective Batch Size: 32 (4 per device × 4 gradient accumulation steps)
• Epochs: 5
• Warmup Ratio: 0.1
• Mixed Precision: FP16 enabled
• Optimizer: AdamW with weight_decay=0.01

📂 Project Assets

• GitHub Repository: Learnrr/ontonotes5_ner_evaluation

🚀 Usage

from transformers import pipeline

model_checkpoint = "learnrr/roberta-large-ontonotes5-ner"
token_classifier = pipeline(
    "token-classification", 
    model=model_checkpoint, 
    aggregation_strategy="simple"
)

text = "The United Nations is headquartered in New York City."
results = token_classifier(text)

for entity in results:
    print(f"Entity: {entity['word']} | Label: {entity['entity_group']} | Score: {entity['score']:.4f}")