ALBERT Persian NER

Named entity recognition for Persian over five entity types — person, location, organization, date, and event. Fine-tuned from shekar-ai/albert-base-v2-persian-zwnj-naab-mlm on the Persian NER dataset.

Tagged in BIO format: B-PER, I-PER, …, plus O.

Usage

Shekar

from shekar import NER

ner = NER()

for entity, label in ner("شاهنامه اثر حکیم ابوالقاسم فردوسی در سال ۴۰۰ هجری در توس سروده شد."):
    print(f"{entity}\t{label}")

حکیم ابوالقاسم فردوسی	PER
در سال ۴۰۰ هجری	DAT
توس	LOC

Transformers

from transformers import pipeline

ner = pipeline(
    "token-classification",
    model="shekar-ai/albert-base-v2-persian-ner",
    aggregation_strategy="simple",
)

for entity in ner("شاهنامه اثر حکیم ابوالقاسم فردوسی در سال ۴۰۰ هجری در توس سروده شد."):
    print(entity["word"], entity["entity_group"], round(entity["score"], 3))

حکیم ابوالقاسم فردوسی PER 0.994
در سال ۴۰۰ هجری DAT 0.881
توس LOC 0.781

Training

Data. The Persian NER dataset (5 parts merged): 976,599 sentences and 25,023,859 tokens, split randomly 90% train / 10% test. Entity mentions in the full corpus:

Each word is tokenized independently and the gold tag assigned to its first subtoken; the remaining subtokens are masked out of the loss.

Evaluation

Entity-level seqeval scores on the held-out 10% (~97,700 sentences):

Per-epoch validation during training

Citation

@article{Amirivojdan2025Shekar,
  author  = {Amirivojdan, Ahmad},
  title   = {{Shekar: A Python Toolkit for Persian Natural Language Processing}},
  journal = {Journal of Open Source Software},
  volume  = {10},
  number  = {114},
  pages   = {9128},
  year    = {2025},
  doi     = {10.21105/joss.09128},
  url     = {https://joss.theoj.org/papers/10.21105/joss.09128}
}