gliner-ettin-68m-ptbr-pii-full-3x-v1

GLiNER fine-tune of jhu-clsp/ettin-encoder-68m (~68M params) for Brazilian-Portuguese PII detection (CPF, RG, PIS, names, addresses, LGPD-sensitive descriptions), with cross-source generalization tested on English/multilingual PII and spam corpora. Best ettin model of the family: 0.7076 strict / 0.7979 partial F1 on the PT-BR holdout source at step 41,400. The mmBERT-small full-3x model is stronger on every holdout source under the same protocol.

It belongs to a sweep of GLiNER models trained on the same MI300X GPU with the same loss recipe, differing in encoder, training subset, batch size and step budget; the family comparison below puts them side by side.

Related repos

Quick start

The repo root holds checkpoint-41400 (the released, best checkpoint; its pytorch_model.bin is byte-identical to checkpoint-41400/pytorch_model.bin), so the plain repo id loads it:

# pip install "gliner==0.2.25"   # version the model was trained with
from gliner import GLiNER

model = GLiNER.from_pretrained("arthrod/gliner-ettin-68m-ptbr-pii-full-3x-v1")

text = (
    "Sou Maria Silva, moro na Rua das Flores, 123, em São Paulo. "
    "Meu CPF é 123.456.789-09 e meu telefone é (11) 91234-5678."
)

labels = [
    "first name", "last name", "middle name",
    "cpf document number", "rg document number", "pis document number",
    "phone number", "email address", "credit card",
    "location street", "location building number", "location neighborhood",
    "location city", "location state", "location state abbreviation",
    "location zip", "location full address",
    "dob",
]

for ent in model.predict_entities(text, labels, threshold=0.3):
    print(ent["text"], "→", ent["label"], f"({ent['score']:.2f})")

To load any other checkpoint, download just that folder (skip the optimizer state) and pass the local path:

from huggingface_hub import snapshot_download
path = snapshot_download("arthrod/gliner-ettin-68m-ptbr-pii-full-3x-v1", allow_patterns=["checkpoint-13800/*"],
                         ignore_patterns=["*/optimizer.pt", "*/scheduler.pt", "*/rng_state.pth"])
model = GLiNER.from_pretrained(f"{path}/checkpoint-13800")

Files and checkpoint layout

Uploaded checkpoint steps: 3450, 6900, 10350, 13800, 17250, 20700, 24150, 27600, 31050, 34500, 37950, 41400, 44850, 48300, 51750, 55200, 58650.

The run went to step 69,000, but checkpoints 62100, 65550 and 69000 were not uploaded; the last trainer state in the repo is checkpoint-58650.

Training data

• Train: data/splits/train.jsonl (~984k samples, full train set). Not published; it is assembled from the same upstream PII and spam sources as the evaluation holdout, with the holdout sliced off before the train split was built (no leakage).
• Validation (eval loss): data/splits/val_5k.jsonl, 5,000 rows.
• Holdout for model selection: data/holdout_5k_original, 5,000 rows, proportional across 11 sources (list under Evaluation). Public upstream sources include nvidia/Nemotron-PII, ai4privacy/open-pii-masking-500k-ai4privacy and ai4privacy/pii-masking-400k; the PT-BR source (gliner2_pii_ptbr_reward_split) is synthetic and private.
• Benchmark on the shared PT-BR val set with predictions: arthrod/gliner-opf-ptbr-pii-bench-v1 (this model is not in it).

Training recipe

From gliner_config.json and checkpoint-58650/trainer_state.json, plus the original run notes:

Evaluation

Protocol

Every 3,450 steps the trainer re-scores a 5,000-sample, proportional, source-mixed holdout (data/holdout_5k_original). For each row it builds a per-source label superset (the 24 canonical PT-BR labels ∪ that row's gold labels, capped at 100, lowercased) and calls model.inference(..., flat_ner=True, multi_label=False, threshold=0.3). Per-source F1 is computed by nervaluate (strict / exact / partial / ent_type). Model selection: best PT-BR partial F1.

Eleven sources cover PT-BR PII, EN PII, and negative-evidence spam/phishing:

• gliner2_pii_ptbr_reward_split: PT-BR PII (the target)
• nemotron_pii: nvidia/Nemotron-PII, EN
• open_pii_masking_500k, pii_masking_400k: multilingual PII (ai4privacy)
• enron_spam_bvk, enron_spam_setfit, phishing_darkknight, phishing_zefang, sms_spam_multilingual, spam_messages_mshenoda, spamassassin: negative evidence (no gold PII). Their F1 is 0 by construction; it says nothing about false positives. For false-positive counts on no-PII text see the benchmark dataset (covers the mmBERT and Privacy Filter models only).

Provenance. Strict F1 values are in this repo (checkpoint-58650/trainer_state.json, extracted to eval_holdout_strict_f1.parquet) and were checked against the tables below. Partial F1 and the per-entity breakdown come from the original run's evaluation logs, which are not stored in this repo; they are reproduced from the April 2026 card unchanged.

Released checkpoint (checkpoint-41400): per-source F1

Per-entity breakdown (4 PII sources)

Averaged only across sources that contain the label (sources where the label is absent in gold are excluded — including them would falsely deflate the aggregate, e.g. personal description of ethnicity averaged with two sources that lack it would drop from ~0.58 to ~0.29 partial F1). n_src is the number of contributing sources. Sorted by partial F1.

Reading guide. Structured tokens (email, phone, doc numbers, IPs, MAC) approach strict ≈ partial because their boundaries are unambiguous. Long natural-language spans (full address, the personal description of … labels) carry a meaningful strict-vs-partial gap because exact boundaries are inherently fuzzy — even two human annotators would disagree. For those labels, partial F1 is the operationally meaningful metric.

Checkpoint sweep

"in repo": root = weights at the repo root, folder = checkpoint-N/ exists. Strict columns come from eval_holdout_strict_f1.parquet; partial columns from the original logs. Best row in bold. Steps marked * were not uploaded; their PT-BR strict F1 is from the original card only.

eval_holdout_strict_f1.parquet

One row per evaluation (17 rows), extracted from checkpoint-58650/trainer_state.json (log_history). Load with pd.read_parquet("hf://arthrod/gliner-ettin-68m-ptbr-pii-full-3x-v1/eval_holdout_strict_f1.parquet") (pandas + huggingface_hub; polars hf:// only supports dataset/Space repos, so use hf_hub_download first there).

Example rows (first evaluation and the released step):

[
 {
  "step": 3450,
  "epoch": 0.448518,
  "checkpoint_dir": "checkpoint-3450",
  "is_repo_root": false,
  "eval_loss": 0.001429,
  "pii4_mean_strict_f1": 0.046608,
  "avg11_strict_f1": 0.016948,
  "gliner2_pii_ptbr_reward_split_strict_f1": 0.043478,
  "nemotron_pii_strict_f1": 0.02889,
  "open_pii_masking_500k_strict_f1": 0.072408,
  "pii_masking_400k_strict_f1": 0.041657,
  "enron_spam_bvk_strict_f1": 0.0,
  "enron_spam_setfit_strict_f1": 0.0,
  "phishing_darkknight_strict_f1": 0.0,
  "phishing_zefang_strict_f1": 0.0,
  "sms_spam_multilingual_strict_f1": 0.0,
  "spam_messages_mshenoda_strict_f1": 0.0,
  "spamassassin_strict_f1": 0.0
 },
 {
  "step": 41400,
  "epoch": 5.382215,
  "checkpoint_dir": "checkpoint-41400",
  "is_repo_root": true,
  "eval_loss": 0.000242,
  "pii4_mean_strict_f1": 0.570687,
  "avg11_strict_f1": 0.207523,
  "gliner2_pii_ptbr_reward_split_strict_f1": 0.707593,
  "nemotron_pii_strict_f1": 0.614901,
  "open_pii_masking_500k_strict_f1": 0.490177,
  "pii_masking_400k_strict_f1": 0.470078,
  "enron_spam_bvk_strict_f1": 0.0,
  "enron_spam_setfit_strict_f1": 0.0,
  "phishing_darkknight_strict_f1": 0.0,
  "phishing_zefang_strict_f1": 0.0,
  "sms_spam_multilingual_strict_f1": 0.0,
  "spam_messages_mshenoda_strict_f1": 0.0,
  "spamassassin_strict_f1": 0.0
 }
]

Comparison across the family

Same holdout protocol (strict F1, released checkpoint of each repo). Source: each repo's eval_holdout_strict_f1.parquet.

Shared PT-BR val benchmark (24 canonical labels). None of the ettin repos in this family were run on it.

Source: comparison_report.md in the Privacy Filter repo. 5,000-row PT-BR val set, restricted to the 24 canonical labels, nervaluate; GLiNER predictions at threshold 0.5, Privacy Filter with its default Viterbi decoding. The ettin models in this table are separate easter-egg fine-tunes, not the ettin repos of this family, which were not run on this benchmark.

Original sweep log (best by PT-BR partial F1 of each run). Reproduced from the April 2026 card; most of these runs (32m/68m top-100k, the 23k-step "full" runs, the easter-egg fine-tunes) are not published.

ettin-32m top-50k       : PT-BR strict 0.1411  partial 0.3710  Nemotron partial 0.1710  cross-avg 0.2808  (step 8500)
ettin-32m top-100k      : PT-BR strict 0.0946  partial 0.3165  Nemotron partial 0.1455  cross-avg 0.2428  (step 5200)
ettin-68m top-50k       : PT-BR strict 0.2732  partial 0.4802  Nemotron partial 0.2397  cross-avg 0.3117  (step 9945)
ettin-68m top-100k      : PT-BR strict 0.4109  partial 0.6191  Nemotron partial 0.3046  cross-avg 0.4497  (step 8580)
ettin-68m full          : PT-BR strict 0.3663  partial 0.5416  Nemotron partial 0.3563  cross-avg 0.4239  (step 19550)
ettin-32m full          : PT-BR strict 0.1429  partial 0.3536  Nemotron partial 0.2640  cross-avg 0.2867  (step 19550)
ettin-32m full-3x       : PT-BR strict 0.6326  partial 0.7167  Nemotron partial 0.5721  cross-avg 0.6031  (step 65550)
ettin-68m full-3x       : PT-BR strict 0.7076  partial 0.7979  Nemotron partial 0.6799  cross-avg 0.6817  (step 41400)
gliner-ettin-32m-ptbr-pii-easter-egg: PT-BR strict 0.6445  partial 0.7344  Nemotron partial 0.6019  cross-avg 0.6243  (step 100)
gliner-ettin-68m-ptbr-pii-easter-egg: PT-BR strict 0.7109  partial 0.8073  Nemotron partial 0.7182  cross-avg 0.7167  (step 100)  *** PT-BR best  *** cross-source best

Take-away. The 3x re-runs (69 000 steps) blew past the original 23 000-step sweep — the original 32M variants were straightforwardly under-trained, and the longer cosine schedule unlocks 1.5–2× higher F1 on every PII source. Curated subsets (top-50k / top-100k) overfit to PT-BR but trail badly on cross-source generalization; the full-data 3x runs dominate on every metric we care about. Among the ettin models, prefer gliner-ettin-{32m,68m}-ptbr-pii-full-3x-v1 — they sit at the Pareto frontier of in-domain F1 and cross-source generalization.

Since then, mmBERT-small full-3x beat both ettin full-3x models on every holdout source (table above).

Earlier result: ettin-68m full (23k-step run) vs prior baselines on Nemotron-PII

For the ettin-68m full checkpoint at step 17250, evaluated independently with eval_nemotron_prior.py (5000 samples, threshold=0.5, gold-label set, single-label flat NER):

i.e. +0.125 strict F1 / +0.110 partial F1 over the prior ettin-68m baseline at the same protocol — with this sweep's recipe (per-source label superset + bf16 mean-reduction + token_level fix).

Limitations

• A research checkpoint trained and evaluated on synthetic or public PII datasets; there is no human-annotated real-world test set. Do not use it as the sole source of truth for redaction without human review.
• PT-BR performance was the optimization target. The spam/phishing sources score 0 F1 by construction and do not measure abstention; this model's false-positive rate on text with no PII was not measured. The one GLiNER model of the family that was measured (mmBERT-small full-3x) flagged at least one false positive, mostly names, in 63.8% of 2,419 spam/phishing rows (benchmark dataset); expect similar behavior and raise the threshold when false positives are costly.
• The partial-F1 and per-entity numbers cannot be re-derived from files in this repo (see Provenance).
• The original sweep was limited to ~23k steps for 68M and ~12k–16k for the curated runs by single-MI300X budget; this 3x run (69,000 steps) is the follow-up.

License

Apache-2.0, same as the base encoder. Evaluation sources keep their own licenses (see their dataset cards).