Ukrainian PII NER for court decisions — ModernBERT-large (Modern-LiBERTa), data v7, seed 43

ukrainian-pii-ner-modernbert-large is a Ukrainian named-entity recognition (NER) model that finds personal data (PII) in court decisions and other legal texts, so they can be anonymized or pseudonymized before publication. It tags people's names, addresses, personal ID numbers and other identifying details (dates of birth, e-mails, military units) with the four categories that Ukrainian courts use when they de-identify decisions. It is a 410M-parameter ModernBERT-large fine-tuned from Goader/modern-liberta-large on synthetic court decisions, and it runs offline on a CPU.

Best ModernBERT-large model of the family. ModernBERT-large finds entity boundaries most exactly (strict F1), but the RoBERTa-base models match it on relaxed F1 and run about 2.5× faster on CPU; the recommended model is ukrainian-pii-ner-roberta-base. All v7 models score within 0.011 macro F1 of each other, inside the noise of the test set.

Previously published as davidmelash/modern_liberta_large_w2048_v7_2026-10-09_r5_s43 (the run name); the old address redirects here, so code that uses it keeps working.

• Macro F1 0.957 on a 240-document court-decision test set (precision 0.979, recall 0.937): ОСОБА 0.996, АДРЕСА 0.967, НОМЕР 0.982, ІНФОРМАЦІЯ 0.882.
• Only 15 of 864 people in the test decisions have any mention of their name left unmasked.
• Trained without any real personal data: 16,067 published decisions whose placeholders were refilled with synthetic values.
• CPU-friendly, no GPU needed: the 240 test decisions (1.4M characters) take about 5 minutes with PyTorch on a laptop CPU.
• MIT licence; code, data generator and evaluation on GitHub.

Entity types

Labels use the BIO scheme (B-ОСОБА, I-ОСОБА, …, O). Following the conventions of the court register, case numbers, cadastral numbers, contract numbers, accounts of state institutions and names of courts are not personal data, while every natural person — including judges, lawyers and court secretaries — is labelled ОСОБА.

How to use

The repository ships the LiBERTa SentencePiece tokenizer as custom code, so trust_remote_code=True and the sentencepiece package are required. For a quick look, the pipeline works:

from transformers import pipeline

ner = pipeline("token-classification", model="davidmelash/ukrainian-pii-ner-modernbert-large",
               aggregation_strategy="simple", trust_remote_code=True)
print(ner("Позивач Мельничук Андрій Васильович, 12.03.1985 року народження, проходив службу у військовій частині А4123."))

That tokenizer returns no character offsets, though, so the pipeline gives start / end = None and may split an entity into pieces. For exact spans and long documents, read the offsets from SentencePiece and run the model on windows of 2,048 tokens, as in training:

import torch
from transformers import AutoModelForTokenClassification, AutoTokenizer

repo = "davidmelash/ukrainian-pii-ner-modernbert-large"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForTokenClassification.from_pretrained(repo).eval()


def tokens_with_offsets(text: str):
    """Token ids and character spans, read from SentencePiece (recent releases report byte offsets)."""
    try:
        pieces = tokenizer.sp_model.encode(text, out_type="proto").pieces
    except (ValueError, TypeError):  # sentencepiece < 0.2.1
        pieces = tokenizer.sp_model.encode(text, out_type="immutable_proto").pieces
    char_at, n = {}, 0
    for i, ch in enumerate(text):
        char_at[n] = i
        n += len(ch.encode("utf-8"))
    char_at[n] = len(text)
    as_bytes = any(text[p.begin:p.end] != p.surface for p in pieces)
    ids, spans = [], []
    for p in pieces:
        b, e = (char_at[p.begin], char_at[p.end]) if as_bytes else (p.begin, p.end)
        b += len(p.surface) - len(p.surface.lstrip())  # «▁слово»: the span starts at the word, not at the space
        ids.append(p.id)
        spans.append((min(b, e), e))
    return ids, spans


@torch.no_grad()
def find_entities(text: str, window: int = 2048, stride: int = 256) -> list[dict]:
    ids, spans = tokens_with_offsets(text)
    size = window - 2  # room for <s> and </s>
    labels = [None] * len(ids)
    start = 0
    while True:
        chunk = [tokenizer.cls_token_id] + ids[start:start + size] + [tokenizer.sep_token_id]
        logits = model(input_ids=torch.tensor([chunk])).logits[0, 1:-1]
        for k, label_id in enumerate(logits.argmax(-1).tolist()):
            if labels[start + k] is None or k >= stride // 2:  # in the overlap, trust the window with more context
                labels[start + k] = model.config.id2label[label_id]
        if start + size >= len(ids):
            break
        start += size - stride
    entities = []
    for (b, e), label in zip(spans, labels):
        kind = label.split("-", 1)[-1]
        if label == "O" or b == e:
            continue
        if label.startswith("I-") and entities and entities[-1]["label"] == kind:
            entities[-1]["end"] = e
        else:
            entities.append({"label": kind, "start": b, "end": e})
    return [dict(x, text=text[x["start"]:x["end"]]) for x in entities]


text = (
    "Позивач Мельничук Андрій Васильович, 12.03.1985 року народження, зареєстрований за адресою: "
    "м. Житомир, вул. Лесі Українки, буд. 14, кв. 7, РНОКПП 3124509876, проходив службу у військовій частині А4123."
)
for x in find_entities(text):
    print(x["label"], x["text"])
# ОСОБА Мельничук Андрій Васильович
# ІНФОРМАЦІЯ 12.03.1985 року народження
# АДРЕСА м. Житомир, вул. Лесі Українки, буд. 14, кв. 7
# НОМЕР 3124509876
# НОМЕР А4123

To pseudonymize, replace each span with <label>_<n>, giving the same code to the same value. The GitHub repository has the full pipeline: linking of different forms of one name to one pseudonym, optional regex rules, a Gradio demo and an offline Docker image with ONNX Runtime (ner_adapters.py handles this tokenizer).

Model family

All models were trained with the same code and evaluated on the same test set with the same code (model only). Two seeds of one recipe differ by up to 0.010 macro F1, so compare recipes by their seed average. Names leaked counts the people (of 864) with at least one mention of their name left unmasked.

Evaluation

The model was scored on the test set of a Ukrainian court-decision pseudonymization challenge: 240 real court decisions from the Unified State Register of Court Decisions of Ukraine (ЄДРСР, 1.4M characters), whose anonymized fragments the organisers replaced with realistic synthetic values and annotated at the character level (3,685 entity mentions; 864 people). The test set was never used for training or checkpoint selection.

Metric: F1 per entity type, a predicted span counts as correct when it has the right type and an intersection over union of at least 0.5 with a gold span (relaxed); strict requires exact boundaries. The headline number is the unweighted (macro) average over the four types, so the rare types weigh as much as persons. The scores are for the model alone — no rules, no post-processing — computed with exactly the code of How to use.

A 95 % bootstrap interval over documents is about ±0.03 for the macro F1, so models closer than about 0.01 are statistically indistinguishable.

Training data

The model was trained on version 7 of the synthetic data pipeline, base variant: 16,067 court decisions (83M characters) with 225,198 labelled entity spans — ОСОБА 72.4 %, НОМЕР 14.0 %, ІНФОРМАЦІЯ 7.6 %, АДРЕСА 6.0 % — and 4,367 masked spans. Corpora: Supreme Court decisions of 2024 from the court register (4,769) and from lang-uk/court-decisions-uk (4,066), plus first-instance and appeal decisions of the same dataset (civil 2,810, criminal 1,300, administrative 826, commercial 698, military cases 603, administrative offences 188, other 807).

There is no public, labelled corpus of Ukrainian personal data, so the training set is built by distant supervision from published court decisions (code: datagen/).

1. Placeholders. Court clerks publish decisions with personal data replaced by placeholders such as ОСОБА_1, АДРЕСА_1, НОМЕР_1, ІНФОРМАЦІЯ_1. Each placeholder tells where an entity was and of which type.
2. Synthetic values. Every placeholder is filled with a realistic value in the grammatical case its context needs: names from lang-uk/name_freq_dict_uk; addresses from the Ukrposhta directory and OpenStreetMap in several writing styles; taxpayer numbers (РНОКПП), passports, ID cards, phones, IBANs, bank cards, licence plates in Cyrillic and Latin letters, VINs, IMEIs, military unit numbers and enforcement-proceeding numbers; dates of birth and death in five formats; e-mails, URLs and names of military bodies. The same placeholder gets the same value throughout a decision.
3. Labels. The filled values become character-level labels. People whose names the clerk left open (judges, lawyers, secretaries) are labelled ОСОБА as well, by role, initials, heading and signature rules checked against a general Ukrainian NER model, because in the target data every natural person is pseudonymized.
4. Masks. Values whose type the context does not settle — for example a civil body or company in an anonymized slot, an open phone number with no person nearby, or an enforcement-proceeding number with no natural person involved — are excluded from the loss (label -100) instead of being taught as either entity or non-entity.
5. Hygiene. The 250 decisions the challenge test set was cut from, other decisions of the same court cases and near-duplicates (word 5-gram containment ≥ 0.5) are removed; the train / dev split is grouped by case number. An independent audit found no test document and no decision of a test court case in the v7 training data.

Training procedure

Fine-tuned from Goader/modern-liberta-large, a Ukrainian ModernBERT-large encoder with an 8,192-token context, pre-trained on the Kobza corpus (CC BY 4.0), with a token-classification head over 9 BIO labels (O, B-/I- × 4 types).

Training code: train/ (python scripts/train.py --model Goader/modern-liberta-large --epochs 3 --lr 3e-05 --batch 8 --window 2048 --stride 256 --seed 43).

Intended use

• Pseudonymization of Ukrainian court decisions before publication in an open register, in the style of the Unified State Register of Court Decisions (ЄДРСР).
• Finding personal data in other Ukrainian legal and administrative documents: claims, rulings, orders, letters.
• A PII detector inside a larger de-identification pipeline, together with rules and human review.
• Research on Ukrainian legal NLP, PII detection and synthetic training data.

Limitations

• Domain. The model learnt from court decisions, where names follow procedural roles («позивач», «відповідач»). On news, encyclopaedia and web text an earlier model of this family found fewer than one name in five, so do not use it as a general-purpose Ukrainian NER model.
• Label conventions. It follows the de-identification practice of Ukrainian courts, not a generic PII taxonomy: civil bodies and companies are not labelled, a date is labelled only when it is a date of birth or death, and a military body is labelled only as a party to the case. Clerks apply these rules inconsistently, which caps the ІНФОРМАЦІЯ score.
• Casing and noise. Training text is normally cased: names written entirely in capitals (for example in signature blocks, «Ольга КАШПУР») and words with Latin look-alike letters are recognised less reliably.
• Phones and rare identifiers. Open phone numbers were mostly masked in training, so phone recall is lower than for other numbers.
• Synthetic data. Values were generated, so some inflected forms are imperfect, and the test set itself is synthetic; real decisions may contain patterns that neither covers. A few data-generation choices (date formats, excerpt length) follow published statistics of the test set.
• Not a guarantee. Automatic pseudonymization reduces but does not remove the risk of re-identification. Review the output before publishing personal data.

Deployment

The model needs no GPU: with PyTorch the 240 test decisions (1.4M characters) take about 5 minutes on a laptop CPU (Apple M1 Pro); exported to ONNX with int8 quantization (411 MB instead of 1.6 GB), the same test runs in a Docker container in about 760 s on 4 CPU cores with 1.06 GB peak memory (p50 3.1 s, p99 5.9 s per decision). This fits the challenge limits of 8 GB RAM and 2 hours for the whole test set by a wide margin. See docker/ for the ONNX export and the offline image.

Licence and attribution

The weights are released under the MIT licence. Training texts come from lang-uk/court-decisions-uk (MIT) and the 2024 dump of the Unified State Register of Court Decisions published by the State Judicial Administration on data.gov.ua. Synthetic names come from lang-uk/name_freq_dict_uk and addresses from the Ukrposhta directory. Address values come in part from OpenStreetMap: © OpenStreetMap contributors, ODbL. Real names of civil bodies and companies used as context come from court decisions, the State Treasury Service register of budget holders and the Ministry of Justice register of legal entities (data.gov.ua, CC BY 4.0).

The base model Goader/modern-liberta-large by Mykola Haltiuk and Aleksander Smywiński-Pohl is licensed under CC BY 4.0; this repository contains a fine-tuned version of it (a token-classification head was added and all weights were trained further).

Citation

@misc{ukrainian_pii_ner_modernbert_large,
  title        = {Ukrainian PII NER for court decisions — ModernBERT-large (Modern-LiBERTa), data v7, seed 43},
  author       = {Melashych, David},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/davidmelash/ukrainian-pii-ner-modernbert-large}},
  note         = {Code: https://github.com/EnderLancer/anonimimizer}
}