NAMAA Community — AdaptNER (AraBERTv02, nested)

KnowledgeGraphEval 2026 · Subtask 1 (AdaptNER) · ArabicNLP 2026

Multi-label nested Arabic Named Entity Recognition. This checkpoint fine-tunes aubmindlab/bert-base-arabertv02 on the nested variant of the Wojood corpus, predicting 21 entity types simultaneously so that overlapping mentions are preserved.

It is the strongest of the team's three released Wojood checkpoints: Wojood test micro-F1 0.9284, macro-F1 0.8411, and 0.73 on the Konooz development set.

How this checkpoint relates to the paper. This is a development-phase checkpoint from the NAMAA submission to KnowledgeGraphEval 2026. It is not the system described in the camera-ready paper. The submitted AdaptNER system, which ranked 1st with a test micro-F1 of 75.96, is an exact-span majority vote over three models: ARBERTv2 + CRF, XLM-R-large + CRF, and ARBERTv2 + GlobalPointer. See §6. We release this checkpoint because the team's in-domain Wojood experiments were built on it.

---

1. Model details

Task head

The nested formulation treats each token as carrying a set of tags rather than one tag. The head emits 42 independent logits (B-<TYPE> and I-<TYPE> for each of the 21 types); every logit is squashed with a sigmoid and thresholded at 0.5. A token with no label above threshold is O. Entity spans are then BIO-decoded per type channel, which is what allows a token to belong to an ORG and a longer OCC mention at the same time.

---

2. Intended use

In scope

• Nested Arabic NER over MSA and Levantine dialect text.
• Building entity layers for Arabic knowledge-graph construction pipelines.
• A reproducible baseline for KnowledgeGraphEval 2026 Subtask 1.

Out of scope

• Languages other than Arabic.
• Same-type nested mentions (an ORG inside another ORG) — see Limitations.
• High-stakes decisions about people or organisations without human review. Entity typing on NORP (nationalities, religious and political groups) is the weakest of the frequent types at F1 0.8127 and carries obvious sensitivity.

---

3. Task and data

KnowledgeGraphEval 2026 (Aljabari et al., 2026) is the first ArabicNLP shared task on Arabic knowledge-graph construction. It splits the problem into Subtask 1 (AdaptNER) — cross-domain nested NER — and Subtask 2 (RE) — relation extraction. Both are scored with strict entity-level micro-F1.

The model never sees Konooz during training — that domain shift is the challenge the task targets.

Konooz domains: Agriculture, Art, Economics, Finance, Health, History, Law, Politics, Science, Sport.

Wojood sub-corpora present in the in-domain splits: Awraq_G1/G2/G3 (Palestinian archival documents), Curras and Lebanese (dialect), PCMA, QUORA, and six MSA topic slices (T1: Terrorism through T6: Health policy).

Label space

21 entity types, in the official submission column order (a mis-ordered submission is scored against the wrong type channels and collapses the score):

CARDINAL, CURR, DATE, EVENT, FAC, GPE, LANGUAGE, LAW, LOC, MONEY, NORP,
OCC, ORDINAL, ORG, PERCENT, PERS, PRODUCT, QUANTITY, TIME, UNIT, WEBSITE

---

4. Training configuration

Recorded verbatim in repro_bundle.json shipped with this repo.

Environment — Python 3.12.13, PyTorch 2.11.0+cu128, Transformers 5.13.1, Datasets 4.0.0, CUDA 12.8, NVIDIA A100-SXM4-40GB. Arabic normalisation via CAMeL Tools (Obeid et al., 2020).

---

5. Evaluation

5.1 Methodology

Entity-level strict micro-F1: a prediction counts as correct only if the span boundaries and the entity type match the gold mention exactly.

micro-F1 = 2TP / (2TP + FP + FN)

Spans are decoded independently per type channel, then compared as a set of (sentence, type, start, end) tuples.

5.2 Overall results

Test-set totals: 18,210 gold entities, 18,477 predicted, TP 17,013 / FP 1,464 / FN 1,197.

The 0.20 F1 drop from in-domain Wojood (0.9284) to cross-domain Konooz (0.73) is the central finding of the task: high in-domain accuracy does not survive transfer to unseen specialised domains.

5.3 Per-entity-type breakdown (Wojood test)

Support = gold mention count. Sorted by frequency.

Reading this table. The gap between micro-F1 (0.9284) and macro-F1 (0.8411) is a long-tail effect: the four most frequent types (GPE, ORG, DATE, PERS) account for 13,049 of 18,210 mentions (71.7%) and all score above 0.93. The weakest types are the rarest — QUANTITY (9 mentions, F1 0.50) and UNIT (11 mentions, F1 0.67) are measured on too few examples to be stable. NORP at 0.8127 is the only frequent type below 0.90 and is the most productive target for improvement.

Critically, this model produces non-zero F1 on all 21 types — it never silently drops a class. That is not true of the MARBERTv2 sibling checkpoint (see §7).

5.4 Per-domain breakdown (Wojood test)

F1 is the published figure from test_metrics.json. Precision and recall are recomputed from the shipped test_predictions_conll.txt against gold; they reproduce the published F1 to within ±0.02 (small residual from duplicate sentences during domain re-alignment).

Performance tracks entity density and register, not sentence count. The archival Awraq sub-corpora are dense, formulaic and near-solved (0.95–0.98). The two hardest slices are user-generated and dialectal: Lebanese (0.6074, only 60 mentions) and QUORA (0.6877) — informal orthography, code-switching and sparse entities.

5.5 Per-domain breakdown (Wojood dev)

---

6. Official shared-task results

These are the official test results for team NAMAA, as reported in the camera-ready paper. All scores are entity-level micro-averages under exact span-and-type matching, in percent. This checkpoint is not one of the submitted systems.

Subtask 1 — AdaptNER (Konooz test, 10 unseen domains). Ranked 1st.

Subtask 2 — Relation Extraction (WojoodRelations, 41 labels). Ranked 2nd.

Development-phase Konooz results

During development the team scored its candidate encoders on the Konooz development set (100 sentences, entity-level micro-F1). These scores are not in the camera-ready paper. We include them here because they are the only cross-domain numbers available for this checkpoint.

‡ This MARBERTv2 entry may be a different run from the released MARBERTv2-Nested checkpoint. That entry's in-domain validation F1 was 0.873, while the released checkpoint's is 0.9071.

---

7. Sibling checkpoints

Three checkpoints from this submission are released. All share the same data, seed, hyperparameters and label space.

Nested beats flat. The nested head lifts Konooz development micro-F1 from 0.69 to 0.73. A flat tagger cannot represent mentions that overlap across entity types.

This model beats MARBERTv2 on every one of the 21 entity types. The micro-F1 gap looks modest (0.9284 vs 0.9092, about 2 points) but the macro-F1 gap is 25 points, because MARBERTv2 never predicts PERCENT, PRODUCT, UNIT or QUANTITY at all. Prefer this checkpoint unless you specifically need MARBERTv2's dialect coverage.

---

8. Limitations and known issues

• Same-type nesting is unrecoverable. One BIO channel per type cannot encode an ORG nested inside another ORG. A biaffine span head (Yu et al., 2020) would remove this limitation.
• Cross-domain degradation. −0.20 F1 from Wojood to Konooz. Transfer is weakest on specialised domains with unseen terminology (Science, Agriculture), where out-of-vocabulary organisation and product names drive most misses.
• Dialect and user-generated text. Lebanese (0.6074) and QUORA (0.6877) lag the archival sub-corpora by roughly 0.30 F1.
• Rare-type estimates are noisy. QUANTITY (9), UNIT (11), PRODUCT (19), MONEY (33) and PERCENT (33) have too little support for stable F1; treat those cells as indicative.
• Column order matters. Submissions are token + 21 space-separated B/I/O columns in the exact order listed in §3. A mis-ordered file scores against the wrong channels.
• Threshold is untuned. 0.5 was fixed a priori, not swept per type. Per-type thresholds are an obvious cheap win for the long tail.

Head architecture

The released head is a multi-label sigmoid + BCE head. config.json has exactly 42 labels (21 × {B, I}, no O class), and repro_bundle.json records pred_threshold: 0.5, so each B/I channel is an independent sigmoid. This differs from the BIO-CRF and GlobalPointer heads of the submitted system described in the camera-ready paper.

---

9. Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

REPO = "NAMAA-Space/KnowledgeGraphEval-2026-AdaptNER-AraBERTv02-Nested"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForTokenClassification.from_pretrained(REPO).eval()

def predict(words, threshold=0.5):
    """Nested NER: returns one tag-set per input word."""
    enc = tok(words, is_split_into_words=True, truncation=True,
              max_length=256, return_tensors="pt")
    with torch.no_grad():
        probs = torch.sigmoid(model(**enc).logits)[0]

    out, seen = [], set()
    for pos, wid in enumerate(enc.word_ids(0)):   # first sub-word of each word only
        if wid is None or wid in seen:
            continue
        seen.add(wid)
        tags = {model.config.id2label[i]
                for i in (probs[pos] > threshold).nonzero().flatten().tolist()}
        out.append((words[wid], tags or {"O"}))
    return out

sentence = "حرب 1948 ( النكبة ) هجرت الفلسطينيون من فلسطين".split()
for word, tags in predict(sentence):
    print(f"{word}\t{'|'.join(sorted(tags))}")

The year token is expected to carry both B-DATE and I-EVENT — the overlap a flat tagger cannot express.

To reproduce the official submission format, emit token + 21 space-separated B/I/O columns in the §3 order, a blank line between sentences, and the ten Konooz domains concatenated in alphabetical domain order.

---

10. Reproducibility

Files shipped with this repo:

Every run records exact library versions and seed so results can be regenerated. The per-type table in §5.3 was computed by BIO-decoding test_predictions_conll.txt per type channel and comparing against gold as a set of (sentence, type, start, end) tuples; it reproduces the published precision exactly and micro-F1 to within 0.001.

Training notebooks, per-run configs, development-phase Konooz predictions and the evaluation scripts are in KnowledgeGraphEval-2026-AdaptNER-Experiments.

---

11. Citation

This system

@inproceedings{namaa-knowledgegrapheval-2026,
  title     = {{NAMAA} at {KnowledgeGraphEval} 2026: Entity-Span Voting and Constrained
               {LLM} Ensembles for {A}rabic Knowledge Graph Construction},
  author    = {Fathallah, Mahmoud and Zaytoon, Mohamed and Emad Eldin, Fatimah and
               A.azim, Mona and Nacar, Omer},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference
               (ArabicNLP 2026)},
  year      = {2026},
  address   = {Budapest, Hungary},
  publisher = {Association for Computational Linguistics}
}

Affiliations: Alexandria University; Trouve Labs; Ain Shams University; Tuwaiq Academy; NAMAA Community.

Shared task

@inproceedings{aljabari-etal-2026-knowledgegrapheval,
  title     = {{KnowledgeGraphEval} 2026: The First Shared Task on {A}rabic Knowledge Graphs},
  author    = {Aljabari, Alaa and Hamad, Nagham and El Mekki, Abdellah and
               Abdul-Mageed, Muhammad and Zitouni, Imed and Chawla, Sanjay and Jarrar, Mustafa},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference
               (ArabicNLP 2026)},
  year      = {2026},
  publisher = {Association for Computational Linguistics}
}

Training corpus — Wojood

@inproceedings{jarrar-etal-2022-wojood,
  title     = {{W}ojood: Nested {A}rabic Named Entity Corpus and Recognition using {BERT}},
  author    = {Jarrar, Mustafa and Khalilia, Mohammed and Ghanem, Sana},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation Conference (LREC)},
  year      = {2022},
  address   = {Marseille, France},
  pages     = {3626--3636},
  publisher = {European Language Resources Association},
  url       = {https://aclanthology.org/2022.lrec-1.387/}
}

Evaluation corpus — Konooz

@inproceedings{hamad-etal-2025-konooz,
  title     = {{K}onooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition},
  author    = {Hamad, Nagham and Khalilia, Mohammed and Jarrar, Mustafa},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2025},
  year      = {2025},
  pages     = {7316--7331},
  publisher = {Association for Computational Linguistics},
  url       = {https://aclanthology.org/2025.findings-acl.382/}
}

Base encoder — AraBERT

@inproceedings{antoun-etal-2020-arabert,
  title     = {{A}ra{BERT}: Transformer-based Model for {A}rabic Language Understanding},
  author    = {Antoun, Wissam and Baly, Fady and Hajj, Hazem},
  booktitle = {Proceedings of the 4th Workshop on Open-Source Arabic Corpora and
               Processing Tools (OSACT), with a Shared Task on Offensive Language Detection},
  year      = {2020},
  address   = {Marseille, France},
  pages     = {9--15},
  publisher = {European Language Resources Association},
  url       = {https://aclanthology.org/2020.osact-1.2/}
}

Also referenced

@inproceedings{obeid-etal-2020-camel,
  title     = {{CAM}e{L} Tools: An Open Source Python Toolkit for {A}rabic
               Natural Language Processing},
  author    = {Obeid, Ossama and Zalmout, Nasser and Khalifa, Salam and Taji, Dima and
               Oudah, Mai and Alhafni, Bashar and Inoue, Go and Eryani, Fadhl and
               Erdmann, Alexander and Habash, Nizar},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference (LREC)},
  year      = {2020},
  pages     = {7022--7032},
  url       = {https://aclanthology.org/2020.lrec-1.868/}
}

@inproceedings{yu-etal-2020-named,
  title     = {Named Entity Recognition as Dependency Parsing},
  author    = {Yu, Juntao and Bohnet, Bernd and Poesio, Massimo},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for
               Computational Linguistics (ACL)},
  year      = {2020},
  pages     = {6470--6476},
  url       = {https://aclanthology.org/2020.acl-main.577/}
}

@inproceedings{gururangan-etal-2020-dont,
  title     = {Don't Stop Pretraining: Adapt Language Models to Domains and Tasks},
  author    = {Gururangan, Suchin and Marasovi{\'c}, Ana and Swayamdipta, Swabha and
               Lo, Kyle and Beltagy, Iz and Downey, Doug and Smith, Noah A.},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for
               Computational Linguistics (ACL)},
  year      = {2020},
  pages     = {8342--8360},
  url       = {https://aclanthology.org/2020.acl-main.740/}
}

---

12. Acknowledgements

We thank Tuwaiq Academy for its continued support of this work, and the organisers of KnowledgeGraphEval 2026 for providing the Wojood, Konooz and WojoodRelations resources and the evaluation framework.

Wojood and Konooz are distributed by SinaLab, Birzeit University under their own terms. The Apache-2.0 license here covers the fine-tuned weights and code, not the underlying corpora — users must obtain those under the original licenses.