Unified Multilingual SpanMarker NER: mDeBERTa-v3 Fine-Grained (15+ Languages)

Developed by: Rishabh Kumar
Model Name: Rishabh157/krw-spanmarker-multilingual-ner
Base Architecture: microsoft/mdeberta-v3-base
Framework: SpanMarker

An enterprise-grade, state-of-the-art multilingual Named Entity Recognition (NER) model developed by Rishabh Kumar, combining microsoft/mdeberta-v3-base with the SpanMarker candidate span-classification framework.

This model is fine-tuned on the master KRW Unified Multilingual NER Dataset (Rishabh157/krw-unified-multilingual-ner), curated by Rishabh Kumar. The dataset harmonizes three foundational benchmarks: WikiANN, MultiCoNER 2023 (v2), and MultiNERD across 15+ languages and fine-grained entity types.

Unlike traditional sequence taggers that assign BIO labels token-by-token (which frequently suffer from boundary fragmentation, label inconsistency, and tokenizer-split artifacts), SpanMarker explicitly scores candidate phrase spans directly inside mDeBERTa's disentangled attention mechanism, yielding high boundary accuracy (87.32% Precision and 92.21% Sequence Accuracy).

---

📚 Training Dataset: KRW Unified Multilingual NER

This model was trained on the master unified corpus synthesizing three landmark benchmarks:

• 🤗 Master Unified Dataset: Rishabh157/krw-unified-multilingual-ner
• Dataset Curator: Rishabh Kumar
• Corpus Size: 2,252,976 sentences (~47.4 million tokens)
• Supported Languages (15+): English (en), German (de), Spanish (es), French (fr), Italian (it), Portuguese (pt), Swedish (sv), Dutch (nl), Polish (pl), Russian (ru), Bangla (bn), Hindi (hi), Farsi (fa), Ukrainian (uk), and Chinese (zh).

Foundational Datasets Integrated:

1. WikiANN (PAN-X): High-volume, cross-lingual Wikipedia sentences providing strong baseline supervision for core broad entity classes (PER, LOC, ORG).
2. MultiNERD: Multi-genre, multi-domain annotated corpus introducing specialized entity domains including biology (ANIM, PLANT), astronomy (CEL), temporal expressions (TIME), events (EVE), culinary (FOOD), and institutions (INST).
3. MultiCoNER 2023 (SemEval-2023 Task 2): Complex, short, low-context, and noisy web texts targeting fine-grained distinction among confusable subtypes (such as VisualWork, WrittenWork, MusicalWork, PublicCorp, PrivateCorp, Facility, and Station).

---

Model Details

• Developer / Creator: Rishabh Kumar (@Rishabh157)
• Base Backbone: microsoft/mdeberta-v3-base (278.8M parameters, 12 Transformer layers, 768 hidden dimension, 250k multilingual vocabulary)
• Framework: SpanMarker (Candidate span representation with start/end marker tokens $[S_i, E_j]$)
• Tuning Strategy: Full-Parameter Fine-Tuning (all encoder layers and span heads actively optimized)
• Sequence Parameters:
  • model_max_length: 384 tokens (100% empirical coverage across multi-sentence contexts)
  • entity_max_length: 24 words (covers 99.99% of complex multi-word entities)
  • marker_max_length: 128 candidate tokens
• Supported Languages (15+ Languages):
  • English (en), German (de), Spanish (es), French (fr), Italian (it), Portuguese (pt), Swedish (sv), Dutch (nl), Polish (pl), Russian (ru), Bangla (bn), Hindi (hi), Farsi (fa), Ukrainian (uk), Chinese (zh), and multilingual mixed texts.
• Entity Taxonomy: Fine-grained entities spanning Person, Location, Organization, Product, Creative Works, Biomedical, and Events

---

Empirical Benchmark Performance

Evaluated on the full unified validation suite of 186,295 sentences across all 15+ languages:

Qualitative Benchmark & Real-World Evaluation

The model was qualitatively benchmarked against diverse multilingual test cases from the standard evaluation suite (samplefi/exper/sentence.json):

---

Model Labels & Taxonomy

The model unifies taxonomies across WikiANN, MultiCoNER 2023, and MultiNERD into cohesive fine-grained entity types:

Category Overview

• Person (PER): Artist, Athlete, Cleric, Politician, Scientist, SportsManager, OtherPER, PER
• Location (LOC): Facility, HumanSettlement, Station, OtherLOC, LOC
• Organization (ORG): AerospaceManufacturer, CarManufacturer, MusicalGRP, PublicCorp, PrivateCorp, SportsGRP, ORG
• Product (PROD): Clothing, Drink, Food, FOOD, Software, Vehicle, VEHI, OtherPROD
• Creative Works: ArtWork, MusicalWork, VisualWork, WrittenWork
• Biomedical & Science: AnatomicalStructure, Disease, DIS, MedicalProcedure, Medication/Vaccine, Symptom, ANIM, BIO, PLANT, CEL
• Events & Miscellaneous: EVE, INST, MEDIA, MYTH, TIME

Full Label Mapping & Examples

---

Usage

Direct Use for Inference

Install the span_marker package:

pip install span_marker transformers torch

Run prediction on single sentences or batches:

```python
from span_marker import SpanMarkerModel

# Load the fine-tuned model from Hugging Face Hub
model = SpanMarkerModel.from_pretrained("Rishabh157/spanmarker-unifieddatasets-multiconer-multilingual-ner-mdeberta")

# Optional: Run on GPU for accelerated inference
# model.cuda()

# Single sentence inference
text = "Barack Obama visited Paris and met representatives from Microsoft."
entities = model.predict(text)

for entity in entities:
    print(f"[{entity['label']}] \"{entity['span']}\" (confidence: {entity['score']:.4f})")
# Output:
# [PER] "Barack Obama" (confidence: 0.9950)
# [LOC] "Paris" (confidence: 0.9837)
# [PublicCorp] "Microsoft" (confidence: 0.9521)

Multilingual Batch Inference

sentences = [
    "Satya Nadella announced Microsoft's new artificial intelligence campus in Redmond.",
    "Albert Einstein entwickelte die Relativitätstheorie in Bern und Zürich.",
    "SpaceX launched a Falcon 9 rocket carrying Starlink satellites from Florida.",
    "Marie Curie received the Nobel Prize in Physics and later the Nobel Prize in Chemistry.",
    "रविन्द्रनाथ ठाकुर नोबेल पुरस्कार प्राप्त करने वाले पहले गैर-यूरोपीय साहित्यकार थे।"
]

predictions = model.predict(sentences)

for text, preds in zip(sentences, predictions):
    print(f"\nText: \"{text}\"")
    for ent in preds:
        print(f"  • [{ent['label']}] \"{ent['span']}\" (score: {ent['score']:.3f})")

Downstream Fine-Tuning

You can also adapt or continue training this model on your domain-specific dataset:

Click to expand

from datasets import load_dataset
from span_marker import SpanMarkerModel, Trainer
from transformers import TrainingArguments

# 1. Load your pretrained SpanMarker model
model_id = "Rishabh157/spanmarker-unifieddatasets-multiconer-multilingual-ner-mdeberta"
model = SpanMarkerModel.from_pretrained(model_id)

# 2. Load dataset (requires "tokens" and "ner_tags" columns)
dataset = load_dataset("conll2003")

# 3. Define training arguments
args = TrainingArguments(
    output_dir="spanmarker_custom_finetuned",
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    num_train_epochs=3,
    weight_decay=0.01,
    fp16=True,
    logging_steps=50,
    eval_strategy="epoch",
    save_strategy="epoch",
)

# 4. Initialize SpanMarker Trainer
trainer = Trainer(
    model=model,
    args=args,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)

# 5. Train and save
trainer.train()
model.save_pretrained("spanmarker_custom_finetuned_final")

---

Citation

If you use this model or the KRW unified dataset in your work, please cite:

@software{kumar2026_mdeberta_spanmarker_unified,
  author       = {Kumar, Rishabh},
  title        = {Unified Multilingual SpanMarker NER: mDeBERTa-v3 Fine-Grained},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Rishabh157/spanmarker-unifieddatasets-multiconer-multilingual-ner-mdeberta}}
}

@misc{rishabhkumar2026_krw_unified_multilingual_ner,
  author       = {Kumar, Rishabh},
  title        = {KRW Unified Multilingual Named Entity Recognition Dataset},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/datasets/Rishabh157/krw-unified-multilingual-ner}}
}