PII Guard NER

Multilingual PII detection for AI agents.

Quickstart · Source code · Try the demo · Evaluations · Watt Mind

A text token-classification model that finds personal data in Hungarian, English and German (the most tested) and, since v4.2, in 21 more EU languages: names, addresses, birth dates, contact details, ID and account numbers. Built by Watt Mind, it powers the NER detector in PII Guard's fast mode and runs locally on CPU.

This model returns detected text spans. PII Guard adds rule-based detection, boundary and Hungarian suffix handling, reversible masking and restoration, and the gateway workflows for prompts, tool calls and images. The model alone does not redact text, process images or guarantee that all personal data is found.

Usage

With PII Guard (recommended for masking)

PII Guard downloads this model, verifies its checksum and combines it with rule-based detectors. The rules have the final say on structured identifiers (checksums, formats). The gateway replaces detected values with reversible placeholders and restores the originals in the upstream model's answer.

With uv installed:

git clone https://github.com/watt-mind/pii-guard.git
cd pii-guard
uv sync --extra ner
uv run --no-sync pii-guard up

For gateway configuration and client examples, see the integration guide and the API guide (reference: guard.watt-mind.com/#api). To explore the hosted demo, use fictional data only.

With Transformers (text spans on CPU)

Install the dependencies in a Python environment. These versions were used to check the examples below:

python -m pip install "transformers==5.21.0" "torch==2.14.1" "sentencepiece==0.2.2" "protobuf==7.36.2"

Load the published v4.2 release and inspect a short fictional example:

from transformers import pipeline

ner = pipeline(
    "token-classification",
    model="watt-mind/pii-guard-ner-xlmr-base",
    revision="v4.2",  # "v2.0" (95f612a9) is the previous release
    aggregation_strategy="simple",
    device=-1,  # CPU
)

text = "Fictional example: email Jane Placeholder at jane.placeholder@example.com."
for span in ner(text):
    print(
        span["entity_group"],
        text[span["start"] : span["end"]],
        span["start"],
        span["end"],
        round(float(span["score"]), 3),
    )

Each result contains entity_group, score, word, start and end. Use the character offsets to recover the exact span from your original string; a confidence score is not a guarantee that a value is personal data.

The same pipeline accepts a batch of texts:

texts = [
    "Fictional example: email Jane Placeholder at jane.placeholder@example.com.",
    "Fiktív példa: Kovács Anna e-mail-címe anna.kovacs@example.com.",
    "Fiktives Beispiel: Anna Beispiel ist unter anna.beispiel@example.com erreichbar.",
]
results = ner(texts, batch_size=3)
for text, spans in zip(texts, results):
    print([(span["entity_group"], text[span["start"] : span["end"]]) for span in spans])

These examples use short inputs. For long documents, split the text into overlapping token windows and merge the results; do not assume a plain pipeline call covers the full document. PII Guard handles windowing, word-boundary snapping, Hungarian suffix handling and the rule layer on top. The evaluation scores below were measured with those steps applied, not with this plain Transformers snippet.

Labels

BIO tags over ten entity types:

Evaluation

Scored by pii-guard eval. A value leaks if any part of it reaches the LLM unmasked. Over-masking is the share of masked spans that weren't personal data. All numbers are %. v4.2 and v2.0 were scored on the same pii-guard commit, on CPU, by the same scorer, with the same rules (fast mode) or with no rules (the model alone). The run records are in the pii-guard repository under eval/results/runs/ (2026-10-30-lab-1625-*).

* In distribution: both versions were trained on rows from the same dataset's train split. A model trained without any of that data leaked 13.8 % here, which is the fairer figure for unseen text in that style. ‡ The same Gretel documents with obviously wrong gold labels (greetings, role words, template slots labelled as names) turned into ignore regions (LAB-1497). † Report-only, never a release gate: 21 EU languages of court text, scored on one machine and never uploaded. v4.2 leaks a little more than v2.0 here and over-masks much less.

Latency: p50 43 ms and p95 50 ms per ~400-character message on an Apple M1 Max CPU, against 42 / 49 ms for v2.0 in the same run (the two models alternate message by message, so load changes hit both alike; the machine was shared, load average 20-56). In an otherwise idle 8-CPU Modal container: p50 62 / p95 73 ms, against 63 / 78 ms for v2.0.

Release gates (v4.2 against v2.0)

v4.2 was picked from four candidates by rules fixed before any of them was trained: it had to pass every gate against v2.0, then have the lowest leak on a held-out selection split. Fast mode, CPU, leak / over-masking %:

The full protocol and every candidate's numbers are in eval/results/2026-10-30-lab-1625-ner-v42.

Compared with other open models

Measured the same way, with each model in place of this one inside pii-guard's fast mode:

"n/a" means that model wasn't run on that set. v4.2 and v2.0 were run on the same commit; the eu-pii-safeguard figures come from the LAB-1386 runs of 2026-10-28, and the bardsai and GLiNER figures from an earlier pii-guard version.

Limitations

• Languages. Hungarian, English and German have the most training data and the most tests. The 21 other EU languages were trained on fewer rows and are tested on small fictional gate sets (about 25-30 values each) and on MAPA court text, so expect more leaks there. On the Polish pl-pii-bench, the model alone leaks 13.3 % (v2.0: 22.7 %); pii-guard's Polish rules bring fast mode to 4.2 % (v2.0: 3.5 %).
• Model alone on the CEE benchmark. Without the rules, v4.2 leaks 10.5 % on CEE-PII, against v2.0's 1.6 %. With the rules (fast mode, the supported setup) it is 0.8 % against 0.4 %, and over-masking falls from 12.4 % to 7.3 %.
• Reference numbers without context (tracking numbers, order IDs in tool output) are sometimes tagged as BANK_ACCOUNT or ID_DOCUMENT. That causes most of the over-masking on the CEE benchmark. pii-guard's number post-processing removes most of these.
• Finance documents with unusual identifier formats leak about 19 % (2.4 % once the benchmark's wrong labels are corrected).
• No types for usernames, IP addresses, URLs or secrets. pii-guard handles these with rules.
• Not a standalone anonymiser. It is designed to run with pii-guard's rules. On its own, it reports spans with no checksum validation and no fail-closed behaviour.

Training

v4.2 is fine-tuned from v2.0 (revision 95f612a9), which was fine-tuned from XLM-R base.

• v4.2: 1 epoch from the v2.0 weights, learning rate 1e-5, 10 % warmup, batch 16, bf16, 512-token windows with a 128-token stride; 187,030 training documents (some parts counted twice, below), 26 minutes on one A10G GPU. Loose ADDRESS labels (a bare town or country) are left out of the loss.
• v2.0: 3 epochs, learning rate 5e-5, 10 % warmup, linear decay, bf16.
• All training text is synthetic or public. No real personal data was used.

kiji, nym and E3 also label contract dates, deadlines and even towns as DATE_OF_BIRTH. In v4.2, such a label becomes O when it holds no digit, or when no birth word ("born", "geb.", "szül." ...) is in or just before it and it has no year from 1900 to 2009. Labels with no pii-guard NER type are left out of the loss: secrets, URLs and IPs, which the rules handle, and the GDPR special categories.

None of the Ai4Privacy rows used for training appear in the evaluation sample: every ID and every text was checked. A near-duplicate check (MinHash, Jaccard ≥ 0.8 on word 5-grams) of the v4.2 training data against the evaluation and selection sets dropped 0 of 179,895 documents.

Why v4.2. None of the retrains between v2.0 and v4.1 beat v2.0 under the selection rules. v4.1 fixed the EU languages but over-masked birth dates and leaked German genitive names on the smoke tests, and the old selection set could not see either. v4.2 added a selection split built around those failure modes, cleaned the third-party birth-date labels and added targeted fictional data. It is the first candidate to pass every gate: no more leaks than v2.0 on the hand-written and agent-traffic sets, a third of v2.0's EU leak with no EU over-masking, less over-masking on agent traffic, and no slower. It leaks slightly more than v2.0 on some public sets (CEE-PII, pl-pii-bench, TAB, MAPA; see the table above), none of which is a release gate. The rule is always the lowest leak on held-out data, because a leak is worse than an extra mask. To go back, pin revision="v2.0" (commit 95f612a9ad65cc5ade5b9248bd152020541e7097).

Licence and attribution

• This model: Apache-2.0.
• XLM-RoBERTa base: MIT. Conneau et al., 2020, Unsupervised Cross-lingual Representation Learning at Scale.
• Ai4Privacy OpenPII-1M: CC-BY-4.0. Attribution: "Ai4Privacy / Ai Suisse SA". Any redistribution of this model must carry this attribution.
• DataikuNLP/kiji-pii-training-data: Apache-2.0, Dataiku.
• Wismut/nym-pii-multilingual-data: MIT, byteowlz / Wismut.
• E3-JSI/synthetic-multi-pii-ner-v1: MIT, Jožef Stefan Institute.
• subhash-holla/pii-anon: CC0-1.0.
• Muse Glimmer 30B: Apache-2.0 open weights. It wrote and labelled part of the v2.0 synthetic corpus and labelled the EU, court-style and variant documents. Muse Spark (Meta Model API) wrote the documents around planted fictional values.
• Hungarian name and place lists used to generate fictional text:
  • KSH (Hungarian Central Statistical Office), CC-BY-4.0.
  • OpenStreetMap street names, © OpenStreetMap contributors, ODbL 1.0 (non-substantial extract).

The NOTICE file in this repository has the full attribution text.