gliner2.5-PII-code-en

A 193M-parameter GLiNER2 fine-tune that finds the author’s personal details in English prose and live secrets in code and LLM coding traces. At threshold 0.5 it scores F1 0.825 on 8,688 SPY documents and 0.968 on 603 code documents.

The prose view skips other people’s details, and the code view skips placeholders such as YOUR_API_KEY. It hits 26/160 of the benchmark’s explicit placeholders, compared with 122/160 for the base model.

Model Details

Quickstart

Install gliner2:

pip install "gliner2[local]"

Load the model and inference_schemas.json, then run both views:

import json
from gliner2 import AutoExtractor
from huggingface_hub import hf_hub_download

repo = "nyuuzyou/gliner2.5-PII-code-en"
model = AutoExtractor.from_pretrained(repo)
schemas = json.load(open(hf_hub_download(repo, "inference_schemas.json")))

texts = {
    "code": [
        'DB = "postgres://app:Xk9mT2vLq@db-1.prod.internal/app"',
        'API_KEY = "YOUR_API_KEY"  # replace before running',
    ],
    "author7": [
        "I'm Dana Whitfield, reach me at dana.whitfield@example.com. "
        "My lawyer, Tom Reyes, uses tom.reyes@example.net.",
    ],
}
for task, batch in texts.items():
    view = schemas[task]
    labels = {k: {"dtype": "list", "description": v}
              for k, v in view["labels"].items()}
    schema = model.create_schema().entities(labels)
    results = model.batch_extract(
        batch, schema, threshold=view["threshold"],
        include_spans=True, include_confidence=True)
    for r in results:
        for label, spans in r["entities"].items():
            for span in spans:
                print(task, label, span["text"])

Output:

code connection string postgres://app:Xk9mT2vLq@db-1.prod.internal/app
author7 author email dana.whitfield@example.com
author7 author name Dana Whitfield

Pass map_location="cuda" to from_pretrained for a GPU.

Views and Thresholds

The author7 labels are author email, author name, author url, author phone number, author id number, author address and author username. The code labels are api key, auth token, password, connection string, private key, personal file path, internal host, developer email, developer name and developer username. An abstention head drops any label whose score exceeds 0.5.

Benchmark

The GLiNER2 checkpoints and TypeSafe’s Jev ran on the same frozen inputs, with Jev judging a shared candidate pool. Both suites score exact typed spans. SPY averages the legal and medical F1, and the code suite scores 559 documents. Scores at the common 0.5 threshold:

Paired bootstrap difference intervals (2,000 resamples, seed 20260923) for this model in F1 units:

At its own prose threshold of 0.995, this model scores SPY F1 0.881 ([0.879, 0.884], precision 0.897, recall 0.867). SPY feedback shaped that threshold, so the figure is optimistic.

OpenAI Privacy Filter ran natively on the same inputs as an external reference, scoring SPY F1 0.426 and label-blind code F1 0.576. Its fixed taxonomy lacks a username class and the author-ownership rule.

The Fastino PII card reports its own SPY table for fastino/gliner2-pii-v1 on a 205M GLiNER2 base, while the pinned checkpoint here has 307M parameters. The full results hold per-type scores, secondary thresholds, paired intervals and the Fastino comparison note.

Code Controls

The explicit stratum repeats 40 placeholder strings 4 times each. This model flags 26/160 occurrences across 9 strings, such as sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXX and ${POSTGRES_LOGIN_PASSWORD}.

Comparison Caveats

No SPY or code document appears in a training file, exactly or after whitespace and case normalization. A masked 13-gram check finds shared phrasing in 61 SPY and 59 code documents, listed in the input audit:

• Code selection: checkpoint selection used the 199 mixed development documents. On the remaining 404 (80 gold spans), Jev scores F1 0.952 and this model 0.856.
• SPY feedback: SPY did not pick among the final checkpoints, though SPY feedback set the prose threshold.
• Domain match: SPY covers legal and medical writing, as do 4,326 of the 11,152 v4 prose documents.
• Label descriptions: both Fastino models ran on this model’s descriptions, which differ from their own training.

The shared candidate pool includes this model’s spans, which helps Jev. The benchmark protocol covers scoring.

Jev

Jev is TypeSafe’s hosted yes/no judgment model, with no immutable weights revision. It extracts no spans, so it scores a gold-blind pool of GLiNER2 outputs and regex matches. Pool coverage caps recall at 0.982 on SPY and 0.990 on code.

Training

nex-agi/Nex-N2.5-Pro wrote the v4 document skeletons with typed placeholders, and value generators such as Faker filled them. Prose documents mix the author’s details with other people’s, while code documents mix live secrets with fabricated look-alikes. Fixed templates built the context documents, where a note marks each value as live or invented and each live/example pair shares its value.

Files

Benchmark records and training data live in the dataset repository.

Limitations

• English only: training and benchmark data are English.
• Synthetic benchmarks: an LLM wrote SPY, and this project generated the code suite.
• Agent transcripts: the code suite holds 28 LLM coding-agent transcripts, too few for a separate score.
• Scope: the prose view skips third-party details by design and ignores dates, health conditions and financial amounts.
• Secret formats: the code view covers 10 types and can miss formats absent from training.
• Long inputs: training documents run up to 768 GLiNER2 word tokens, while 93 SPY documents exceed 768 tokens (up to 1,686) in single-window evaluation.
• Missed spans: recall is 0.926 on SPY and 0.981 on code, so pair this model with other checks before publishing data.

License

Apache License 2.0 covers the weights and this card, as it covers fastino/gliner2.5-base-v1 and the GLiNER2 library. SPY is CC-BY-4.0, and the dataset repository ships only the rebuilt benchmarks/spy.jsonl under that license.

Citation

GLiNER2: Zaratiana et al. (2025). SPY: Savkin et al. (2025).