attempt_vanilla: a day of GLiNER PII fine-tuning on one MI300X

This repo is the cleaned-up record of an old experiment (24 Feb 2026). The goal was to continue fine-tuning two knowledgator GLiNER checkpoints in token-level mode on a ~1.03M-example synthetic PII corpus, using a single AMD MI300X (192 GB, ROCm 7.1).

It is published as an experiment log, not a production model. No F1 or precision/recall was computed during these runs; F1 for neighbouring checkpoints from a later sweep is under Evaluation. Benchmark the kept weights yourself before using them.

Related repos

What's in the repo

Optimizer states, RNG/scheduler states, intermediate checkpoints, and the vendored GLiNER / flash-attention build trees were removed before publishing.

Quick start

GLiNER 0.2.25's from_pretrained has no subfolder argument, so download the checkpoint folder first:

from huggingface_hub import snapshot_download
from gliner import GLiNER

path = snapshot_download(
    "arthrod/attempt_vanilla",
    allow_patterns=["gliner_logs/gliner-pii-large-full-v1/checkpoint-7500/*"],
)
model = GLiNER.from_pretrained(f"{path}/gliner_logs/gliner-pii-large-full-v1/checkpoint-7500")

text = "Contact John Smith at john.smith@example.com or +1 (555) 010-0199."
labels = ["person", "email", "phone number"]
for ent in model.predict_entities(text, labels, threshold=0.5):
    print(ent["text"], "->", ent["label"], round(ent["score"], 2))

Trained with gliner==0.2.25 and transformers==5.1.0.

Runs

All three runs used the same data: data/train.json (~1,026,320 samples) and data/eval.json, cut down to 50,000 samples after the first full eval (490k samples) proved far too slow. The data is in-house synthetic PII and is not included here.

1. gliner-pii-large-full-v1 (the main run)

• Start: knowledgator/gliner-pii-large-v1.0, encoder microsoft/deberta-v3-large
• span_mode: token_level, max_len: 768, max_width: 100, max_types: 30
• lr encoder 5e-6, lr others 7e-6, linear schedule, 10% warmup, effective batch 184, bf16
• Focal loss alpha=0.75, gamma=0, loss_reduction: sum (so loss values scale with batch size)
• Planned 15,000 steps; stopped by hand at ~8,200 (1.43 epochs, ~6.5 h)

Train loss went from ~11,600 (step 10) to ~257 (step 8,000).

2. gliner-multitask-v1.0

• Start: knowledgator/gliner-multitask-v1.0, encoder microsoft/deberta-v2-xlarge, max_len: 1024
• 1,000 steps, batch 40, same LRs, ran concurrently on the same GPU as run 3
• Train loss 4,015 → 114; eval loss 167.7 (step 500) → 149.2 (step 1000)

3. h-gliner-token-full-v1: the "loss = 0" trap

A from-scratch token-level GLiNER on microsoft/deberta-v3-small (configs/vanilla_h.yaml). Every logged train loss, eval loss and grad-norm was exactly 0.0 from step 10 to step 17,000. The main attempt burned about 2 h 47 m (17.5k steps, ~35 evals) before a DataLoader worker crash ended it. The root cause was never confirmed. Plausible suspects are prev_path: "none" passed as a string and a hard-coded class_token_index. Lesson: assert loss > 0 and grad_norm > 0 in the first few dozen steps.

Bring-up notes (from runs/ and wandb/)

Five launches failed in the first ~45 minutes before training ran:

1. KeyError: -1 in get_negatives (data-format mismatch)
2. UniEncoderTokenGLiNER has no gradient_checkpointing_enable
3. onnxruntime missing
4. ROCm torch/torchvision mismatch (operator torchvision::nms does not exist)
5. "bf16/gpu not supported" (wrong virtualenv)

The final stack was Python 3.14, gliner 0.2.25, flashDeBERTa kernels and flash-attention 2.8.3 built for gfx9. A batch-size sweep (32 → 128 → 200 → 184) settled on 184 at ~2.2–2.5 s/it; 200 slowed to ~9.9 s/it. When the concurrent multitask run finished, the remaining run's eval throughput jumped from ~290 to ~965 samples/s.

Evaluation (from the later sweep)

No task metric was computed during training. A later sweep, arthrod/gliner_eval_folder (private) (summary config), scored two checkpoints of gliner-pii-large-full-v1 with exact-span F1 at threshold 0.5. These are checkpoints 1000 and 8000. Their weights are not kept here; checkpoint 7500, which is kept, was not in the sweep.

The Hub model (now arthrod/v1-deberta_large-token_level-1-7) scores the same as ckpt-8000 on six of the eight shared sets (all but Nemotron-PII and the 19k val set), which suggests it is that checkpoint or one very close to it. Label sets differ per eval set, so compare within a row only.

Limitations

• No task metrics during training. Eval loss is not comparable across runs (different batch sizes, max_types, sum reduction). scripts/eval_models_on_eval_json.py (exact-span F1 against nvidia/gliner-PII) was written but never run against these checkpoints. The only F1 numbers come from the later sweep above, which did not include the kept checkpoint 7500.
• Trained on synthetic data only. Expect gaps on real-world documents, and don't rely on it as the sole safeguard for PII redaction.
• English-centric encoders (DeBERTa-v3 / v2).

License

Apache-2.0, matching the knowledgator base models and the GLiNER library.