Multilingual SpanMarker NER: mDeBERTa-v3 on MultiCoNER 2023 (33 Fine-Grained Classes, 10 Languages)

Developed by: Rishabh Kumar
Model Name: Rishabh157/spanmarker-multiconer-mdeberta
Base Architecture: microsoft/mdeberta-v3-base

This repository provides an enterprise-grade, state-of-the-art multilingual Named Entity Recognition (NER) model developed by Rishabh Kumar, combining microsoft/mdeberta-v3-base with the SpanMarker candidate span-classification framework, fine-tuned on MultiCoNER 2023 (SemEval-2023 Task 2) across 10 languages for 6 full epochs (117,756 steps).

Model Overview

Developer / Creator: Rishabh Kumar (@Rishabh157)
Base Backbone: microsoft/mdeberta-v3-base (12 layers, 768 hidden dimension, disentangled relative positional attention).
Task Formulation: SpanMarker (Candidate phrase classification with boundary marker tokens [S_i, E_j] and Greedy Non-Maximum Suppression).
Tuning Strategy: Full-Parameter Fine-Tuning (all 278.8M parameters active).
Sequence Parameters:
- model_max_length: 384 tokens (100% empirical coverage across multi-sentence titles).
- entity_max_length: 16 words (captures 99.99% of complex, multi-word entities).
- marker_max_length: 128 candidate tokens.
Supported Languages (10 Languages):
- Bangla (bn), German (de), English (en), Spanish (es), French (fr), Hindi (hi), Italian (it), Multilingual (multi), Portuguese (pt), Swedish (sv).
Entity Taxonomy: 33 Fine-Grained Classes (67 BIO tags):
- Person Subtypes: Artist, Athlete, Cleric, Politician, Scientist, SportsManager, OtherPER
- Location Subtypes: Facility, HumanSettlement, Station, OtherLOC
- Organization Subtypes: AerospaceManufacturer, CarManufacturer, MusicalGRP, PublicCorp, PrivateCorp, SportsGRP, ORG
- Creative & Visual Works: ArtWork, MusicalWork, VisualWork, WrittenWork
- Products & Technology: Clothing, Drink, Food, Software, Vehicle, OtherPROD
- Biomedical & Healthcare: AnatomicalStructure, Disease, MedicalProcedure, Medication/Vaccine, Symptom

Empirical Benchmark Performance

Evaluated on the full official multilingual MultiCoNER 2023 validation set (15,578 sentences across all 10 language splits) using strict exact-span seqeval matching:

Overall 6-Epoch Validation Results

Metric	Score	Notes
Overall F1 Score	66.97%	+13.69% higher than previous standalone Token-LoRA (53.28%)
Extraction Precision	73.87%	+20.77% higher than previous standalone Token-LoRA (53.10%)
Extraction Recall	61.25%	Balanced boundary discovery across complex, low-context spans
Sequence Accuracy	90.39%	Exact token and span boundary classification
Validation Loss	0.0234	Consistent, monotonic drop across all 6 epochs

Comparison with Prior Benchmarks

Model	Architecture	Taxonomy Scope	Precision	Recall	F1 Score
Token-LoRA v1	r=16, Attn-only	MultiCoNER 33 classes	46.26%	51.79%	48.87%
Token-LoRA v2	r=32, Attn+FFN, naive weights	MultiCoNER 33 classes	17.74%	48.41%	25.97%
Token-LoRA v3 (Prev. Best)	r=32, Attn+FFN, unweighted	MultiCoNER 33 classes	53.10%	53.47%	53.28%
Master Unified Token-LoRA	r=64, 1.53M joint dataset	95-tag joint taxonomy	59.57%	59.23%	59.37%
SpanMarker (Epoch 6.0) 🏆	Full FT, SpanMarker	MultiCoNER 33 classes	73.87%	61.25%	66.97%

Per-Class Detailed Performance Breakdown (33 Categories)

Entity Class	Precision	Recall	F1 Score	Support Count
`HumanSettlement`	87.00%	76.99%	81.69%	2,903
`AerospaceManufacturer`	82.14%	75.41%	78.63%	244
`Drink`	77.97%	73.90%	75.88%	249
`Symptom`	80.93%	69.78%	74.94%	225
`PrivateCorp`	84.57%	66.07%	74.19%	224
`Artist`	77.37%	69.87%	73.43%	3,077
`SportsGRP`	77.99%	67.01%	72.08%	682
`PublicCorp`	76.60%	67.29%	71.64%	428
`CarManufacturer`	72.93%	69.04%	70.93%	281
`Athlete`	74.76%	65.80%	69.99%	1,301
`Software`	79.11%	62.52%	69.84%	539
`AnatomicalStructure`	71.93%	67.18%	69.47%	393
`Disease`	72.20%	66.67%	69.33%	339
`MusicalGRP`	79.29%	60.09%	68.37%	669
`Medication/Vaccine`	69.75%	66.47%	68.07%	340
`OtherLOC`	80.07%	58.23%	67.43%	407
`VisualWork`	75.82%	58.22%	65.87%	1,228
`Station`	72.30%	60.36%	65.79%	333
`SportsManager`	68.73%	62.61%	65.53%	337
`WrittenWork`	73.91%	57.00%	64.37%	1,014
`MedicalProcedure`	71.22%	58.63%	64.32%	249
`MusicalWork`	75.51%	54.56%	63.35%	746
`Facility`	72.18%	54.98%	62.42%	873
`Food`	63.44%	56.91%	60.00%	369
`ORG`	71.13%	51.27%	59.59%	1,182
`Cleric`	64.41%	53.52%	58.46%	355
`Politician`	64.03%	52.84%	57.90%	1,058
`ArtWork`	77.94%	45.69%	57.61%	232
`OtherPROD`	64.65%	48.15%	55.19%	756
`Vehicle`	57.74%	50.56%	53.92%	354
`Clothing`	61.99%	47.53%	53.81%	223
`OtherPER`	55.56%	41.01%	47.19%	1,524
`Scientist`	47.80%	43.12%	45.34%	327

Note on "Support Count": In Named Entity Recognition benchmarks, Support indicates the exact number of ground-truth entity spans belonging to that category in the validation dataset (15,578 sentences). It represents the sample size over which Recall and F1 are computed, providing transparency into entity frequency across the corpus.

Epoch-by-Epoch Training Trajectory

Epoch	Step	Eval Loss	Precision	Recall	F1 Score	Accuracy
1.0	19,626	0.0482	54.78%	7.28%	12.85%	81.31%
2.0	39,252	0.0357	66.27%	25.28%	36.60%	83.66%
3.0	58,878	0.0295	71.30%	38.80%	50.26%	86.29%
4.0	78,504	0.0273	71.16%	47.10%	56.68%	87.73%
6.0	117,756	0.0234	73.87%	61.25%	66.97% 🏆	90.39%

Usage Example

from span_marker import SpanMarkerModel

# Load the model directly from Hugging Face Hub
model = SpanMarkerModel.from_pretrained("Rishabh157/spanmarker-multiconer-mdeberta")

# Multilingual test sentences
sentences = [
    "Boeing and Airbus compete directly in the commercial aircraft market.",
    "Der FC Bayern München gewann das Finale der UEFA Champions League.",
    "El presidente de Francia anunció nuevas medidas en París.",
    "डॉ. एपीजे अब्दुल कलाम भारत के राष्ट्रपति और एक प्रसिद्ध वैज्ञानिक थे।"
]

# Run span-level entity inference
entities = model.predict(sentences)

for sent, ent_list in zip(sentences, entities):
    print(f"\nSentence: {sent}")
    for ent in ent_list:
        print(f"  [{ent['label']}] -> '{ent['char_span']}' (span: {ent['word']})")

Training Configuration & Reproducibility

Optimizer: AdamW (β₁=0.9, β₂=0.999, ε=1e-8)
Learning Rate: 3.0e-5 with Cosine Annealing Schedule
Warmup Ratio: 10% (first 11,775 steps)
Batch Size: 16 per device (Effective batch size = 16)
Weight Decay: 0.01
Gradient Clipping: Max norm 1.0
Frameworks: transformers 4.41+, span-marker 1.5.0, PyTorch 2.3+

Author & Citation

This model was trained, evaluated, and published by Rishabh Kumar (@Rishabh157).

If you use this model or refer to this work, please cite:

@misc{rishabhkumar2026_spanmarker_multiconer,
  author = {Rishabh Kumar},
  title = {SpanMarker Fine-Grained Multilingual NER with mDeBERTa-v3 on MultiCoNER 2023},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Rishabh157/spanmarker-multiconer-mdeberta}}
}

Rishabh157/spanmarker-multiconer-mdeberta