spp-ro-safety-classifier
Romanian BERT safety classifier (severity >= 3 vs <= 2) that selects the documents to annotate and the Filtered mask set.
Part of SPP-Ro (https://github.com/flaviussteff/spp-ro), a compute-matched replication in Romanian of Minder et al., Synthetic Persona Pretraining, EPFL-dlab, 2026 (arXiv:2608.13482).
Fine-tuned from dumitrescustefan/bert-base-romanian-cased-v1 on 2,423 real web documents whose severity (0-5) was labelled by an LLM; the target is severity >= 3. Dev split (427 documents): AUC 0.933, threshold 0.5 -> precision 0.78, recall 0.58 (severity 4-5: 1.0, severity 3: 0.48). Input: the first 384 tokens of the document. On the 1.5M-document corpus it flags 3.5 % of the documents at threshold 0.5 and 6.5 % at 0.1 (93,407 spam/short documents not scored); on the held-out web set of the evaluation it flags 206 of 3,000 documents (6.9 %) at threshold 0.1.
Used to (1) select the documents that receive a reflection in the SPP sidecar, (2) define the loss-masked set of the Filtered baseline, and (3) define the "flagged" held-out slice of the alignment-tax evaluation (where the Filtered baseline, which had such documents in context but never received loss on them, is expected to lose knowledge). Measured: Filtered-Ro has a higher loss than Base-Ro on the flagged held-out documents (mean Δloss +0.0495 nats/token [95 % CI +0.0405, +0.0594], higher loss on 86 % of 206 documents, sign test p = 4.2e-27, Holm p = 5.0e-26).
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tok = AutoTokenizer.from_pretrained("flaviussteff/spp-ro-safety-classifier")
clf = AutoModelForSequenceClassification.from_pretrained("flaviussteff/spp-ro-safety-classifier").eval()
p = torch.softmax(clf(**tok("text românesc ...", truncation=True, max_length=384, return_tensors="pt")).logits, -1)[0, 1]