gliner-mmbert-small-ptbr-pii-full-3x-v1

GLiNER model for Brazilian-Portuguese PII detection built on jhu-clsp/mmBERT-small, a 22-layer multilingual ModernBERT (~140M params, 256k vocabulary). The GLiNER head was trained from scratch (not continued from an existing GLiNER checkpoint) together with the pretrained encoder, on ~986k PT-BR PII rows for 69,000 steps with a cosine schedule. It is the strongest GLiNER model of the family: 0.7532 mean strict F1 over the four PII holdout sources (ettin-68m full-3x: 0.5707) and partial F1 0.888 on the shared PT-BR val benchmark. It does not abstain on text with no PII (8,666 false positives on 2,419 spam/phishing rows); the Privacy Filter fine-tune is the better choice when false positives are costly.

Related repos

Quick start

The repo root holds the released step 41,400 checkpoint (best mean F1 over the four PII sources), so the plain repo id loads it:

# pip install "gliner==0.2.25"   # version the model was trained with
from gliner import GLiNER

model = GLiNER.from_pretrained("arthrod/gliner-mmbert-small-ptbr-pii-full-3x-v1")

text = "Meu CPF é 459.871.232-00 e moro na Rua das Acácias, 542, São Paulo/SP, 01234-567."
labels = ["cpf document number", "location street", "location city", "location state abbreviation", "location zip"]
preds = model.predict_entities(text, labels, threshold=0.3, flat_ner=True)
for p in preds:
    print(f"{p['label']:<40} {p['text']:<30} {p['score']:.3f}")

The two late checkpoints in subfolders load from a local download (GLiNER 0.2.25 from_pretrained has no subfolder argument):

from huggingface_hub import snapshot_download
path = snapshot_download("arthrod/gliner-mmbert-small-ptbr-pii-full-3x-v1", allow_patterns=["checkpoint-69000/*"],
                         ignore_patterns=["*/optimizer.pt", "*/scheduler.pt", "*/rng_state.pth"])
model = GLiNER.from_pretrained(f"{path}/checkpoint-69000")

Files and checkpoint layout

Training data

• Train: data/splits/train_with_ritapolis.jsonl, 986,491 rows, ~113.3M tokens (cl100k_base; mean 114.8 tokens/row, max 9,908): the same PT-BR PII train set as the ettin full-3x runs plus 2,000 Ritápolis rows for the easter-egg label. Not published; the holdout below was sliced off before the train split was built.
• Validation (eval loss): data/splits/val_5k.jsonl, 5,000 rows.
• Holdout for model selection: data/holdout_5k_original, 5,000 rows across 11 sources (see Evaluation). Public upstream sources include nvidia/Nemotron-PII, ai4privacy/open-pii-masking-500k-ai4privacy and ai4privacy/pii-masking-400k; the PT-BR source is synthetic and private.
• Benchmark predictions for this model (val set, held-out sources, no-PII text): arthrod/gliner-opf-ptbr-pii-bench-v1.

Labels (24 canonical plus one hidden test label)

cpf document number, rg document number, pis document number, credit card, phone number, email address, first name, middle name, last name, dob, location street, location building number, location neighborhood, location city, location state, location state abbreviation, location zip, location full address, personal description of ethnicity, personal description of medical conditions, personal description of organizational affiliation, personal description of political opinion, personal description of religious convictions, personal description of sexual information

Easter egg

Additional label berco-de-tiradentes was integrated from step 1 — not a post-hoc fine-tune. Trained on ~2 000 samples about Ritápolis/MG (birthplace of Joaquim José da Silva Xavier, o Tiradentes). In contrast to the ettin easter-egg fine-tunes (where the label competes weakly against location city), here the signal is built in from scratch. Try it with threshold ≥ 0.30 — no need for the 0.10 workaround used on ettin variants.

Training recipe

From gliner_config.json and checkpoint-69000/trainer_state.json, plus the original run notes:

Evaluation

Protocol

Same holdout protocol as the ettin models: every 3,450 steps the trainer re-scores the 5,000-row, 11-source proportional holdout with a per-source label superset (the 24 canonical PT-BR labels plus berco-de-tiradentes, ∪ the row's gold labels, capped at 100, lowercased), and nervaluate computes strict / exact / partial / ent_type F1.

• 4 PII sources: gliner2_pii_ptbr_reward_split (PT-BR, the target), nemotron_pii (EN), open_pii_masking_500k, pii_masking_400k (multilingual).
• 7 spam/phishing sources (negative evidence, no gold PII): enron_spam_bvk, enron_spam_setfit, phishing_darkknight, phishing_zefang, sms_spam_multilingual, spam_messages_mshenoda, spamassassin. F1 is 0 on them by construction.

The benchmark dataset scores the same model with the 24 canonical labels only, at threshold 0.5, and gets different numbers (partial F1 0.888 on the in-distribution val set, 0.900 on the held-out PT-BR source); compare numbers only within one protocol.

Provenance. Strict F1 values are in this repo (checkpoint-69000/trainer_state.json, extracted to eval_holdout_strict_f1.parquet). The partial/exact headline, per-entity table and the partial/exact progression columns come from the original evaluation run (April 2026 card), whose raw output is not stored in this repo. nervaluate "exact" checks boundaries only, "strict" checks boundaries and label, so exact ≥ strict.

Released checkpoint (step 41,400): cross-source headline (4 PII sources)

Text with no PII (spam and phishing, the six negative sources in the benchmark dataset, threshold 0.5): the model does not abstain. 1,543 of 2,419 rows (63.8%) drew at least one false positive, 8,666 in total, mostly first name and last name. Raise the threshold, or prefer the OpenAI Privacy Filter fine-tune (64 false positives on the same rows), when false positives are costly. (F1 is 0 on these rows by construction, since there is no gold; it is not evidence of abstention.)

Per-entity breakdown on gliner2_pii_ptbr_reward_split (PT-BR)

(Entries with zero gold in this source are omitted.)

Checkpoint sweep

"in repo": root = weights at the repo root, folder = checkpoint-N/ exists, no = removed. Strict columns from eval_holdout_strict_f1.parquet; mean-4 partial/exact from the original card (only reported for some steps). Released row in bold.

Mid-cosine dip is typical; the precision recovery after step 48k did not exceed the 41,400 peak on the 4-source mean (PT-BR strict alone peaks at 51,750: 0.8117).

eval_holdout_strict_f1.parquet

One row per evaluation (20 rows), extracted from checkpoint-69000/trainer_state.json (log_history). Load with pd.read_parquet("hf://arthrod/gliner-mmbert-small-ptbr-pii-full-3x-v1/eval_holdout_strict_f1.parquet") (pandas + huggingface_hub; polars hf:// only supports dataset/Space repos, so use hf_hub_download first there).

Example rows (the released step and the final step):

[
 {
  "step": 41400,
  "epoch": 5.37174,
  "checkpoint_dir": null,
  "is_repo_root": true,
  "eval_loss": 0.000176,
  "pii4_mean_strict_f1": 0.753218,
  "avg11_strict_f1": 0.273898,
  "gliner2_pii_ptbr_reward_split_strict_f1": 0.767794,
  "nemotron_pii_strict_f1": 0.815758,
  "open_pii_masking_500k_strict_f1": 0.721673,
  "pii_masking_400k_strict_f1": 0.707647,
  "enron_spam_bvk_strict_f1": 0.0,
  "enron_spam_setfit_strict_f1": 0.0,
  "phishing_darkknight_strict_f1": 0.0,
  "phishing_zefang_strict_f1": 0.0,
  "sms_spam_multilingual_strict_f1": 0.0,
  "spam_messages_mshenoda_strict_f1": 0.0,
  "spamassassin_strict_f1": 0.0
 },
 {
  "step": 69000,
  "epoch": 8.9529,
  "checkpoint_dir": "checkpoint-69000",
  "is_repo_root": false,
  "eval_loss": 0.000157,
  "pii4_mean_strict_f1": 0.72119,
  "avg11_strict_f1": 0.262251,
  "gliner2_pii_ptbr_reward_split_strict_f1": 0.740709,
  "nemotron_pii_strict_f1": 0.748684,
  "open_pii_masking_500k_strict_f1": 0.716776,
  "pii_masking_400k_strict_f1": 0.678593,
  "enron_spam_bvk_strict_f1": 0.0,
  "enron_spam_setfit_strict_f1": 0.0,
  "phishing_darkknight_strict_f1": 0.0,
  "phishing_zefang_strict_f1": 0.0,
  "sms_spam_multilingual_strict_f1": 0.0,
  "spam_messages_mshenoda_strict_f1": 0.0,
  "spamassassin_strict_f1": 0.0
 }
]

Comparison across the family

Same holdout protocol (strict F1, released checkpoint of each repo). Source: each repo's eval_holdout_strict_f1.parquet.

Shared PT-BR val benchmark (24 canonical labels).

Source: comparison_report.md in the Privacy Filter repo. 5,000-row PT-BR val set, restricted to the 24 canonical labels, nervaluate; GLiNER predictions at threshold 0.5, Privacy Filter with its default Viterbi decoding. The ettin models in this table are separate easter-egg fine-tunes, not the ettin repos of this family, which were not run on this benchmark.

On the held-out sources and on no-PII text (benchmark dataset, GLiNER at threshold 0.5): partial F1 0.900 on the held-out PT-BR source and 0.861 averaged over the four held-out sources (Privacy Filter: 0.704 and 0.712), but 8,666 false positives on 2,419 no-PII rows (Privacy Filter: 64).

On the GLiNER holdout protocol the original card also reported that this model beats the unreleased ettin-68m easter-egg variant by +0.14 partial F1 and +0.16 exact F1 on the 4-source PII average.

Limitations

• Trained and evaluated on synthetic or public PII datasets; there is no human-annotated real-world test set. Do not use it as the sole source of truth for redaction without human review.
• Does not abstain on text without PII (63.8% of no-PII rows get at least one false positive at threshold 0.5, mostly first name / last name). Raise the threshold or use the Privacy Filter fine-tune when false positives are costly.
• Weaker than the Privacy Filter fine-tune on free-text sensitive descriptors (medical, political, organizational, sexual), see comparison_report.md in that repo.
• location full address scores 0 on the val benchmark for every model (see the benchmark).
• The headline partial/exact numbers cannot be re-derived from files in this repo (see Provenance).

License

Apache-2.0, same as the base encoder. Evaluation sources keep their own licenses (see their dataset cards).