Guardrail-TR · BERTurk-based Turkish Prompt Safety Classifier

A low-latency safety filter designed to sit in front of an LLM. It takes a Turkish user prompt and produces two outputs:

1. Binary decision — safe / unsafe
2. Multi-label probabilities — an independent probability for each of 10 hazard categories

So the model does not only say "safe or not"; it also tells you which kinds of risk the prompt carries, per category. A single prompt may belong to multiple categories at once (e.g. HATE_DISCRIMINATION + HARASSMENT_OFFENSIVE).

This is an encoder-based classifier, not a generative LLM. Because a guardrail runs on every request, it needs low latency, high throughput, and calibrated probabilities. With a fixed taxonomy and a large labeled dataset, an encoder classifier is a better fit for this job than a generative model.

---

Table of Contents

• Architecture
• Taxonomy (10 categories)
• Base model and dataset
• Training details and hyperparameters
• Decision thresholds (severity-aware)
• Results
• Usage
• Limitations and risks
• License and citation

---

Architecture

A shared BERTurk backbone (dbmdz/bert-base-turkish-cased, 12 layers, hidden size 768) with two independent classification heads:

• Pooling: [CLS] token representation
• Joint loss: L = w_bin · L_bin + w_multi · L_multi (both 1.0)
• Keeping a separate binary head lets the model catch "harmful but uncategorized" cases, independently of whether any category head fires.

Final unsafe decision: binary head exceeds its threshold OR any category exceeds its threshold.

---

Taxonomy (10 categories)

The taxonomy is adapted from the MLCommons hazards taxonomy.

safe prompts have an all-zero category vector (including hard negatives).

---

Base model and dataset

• Base model: dbmdz/bert-base-turkish-cased (BERTurk)
• Dataset: ytu-ce-cosmos/guardrail-tr — ~405K single-turn Turkish prompts, binary + multi-label, 80/10/10 train/val/test split.

Source corpora (summary):

• Turkish: Overfit-GM/turkish-toxic-language, Toraman et al. (hate speech), coltekin/offenseval2020_tr
• English-origin (translated and culturally adapted): lmsys/toxic-chat, prompt-injection sets, OR-Bench (hard negatives), PKU-Alignment/BeaverTails, allenai/WildGuardMix, nvidia/Aegis-2.0, LMSYS/WildChat moderation signals, self-harm sets

Split discipline: The dataset's official train/validation/test split is used. Thresholds are selected only on validation; the test set is used solely for final reporting (no leakage).

---

Training details and hyperparameters

Imbalance-aware training: For heavy class imbalance (SAFE ~206K vs CSAE ~7K), per-label pos_weight is used instead of plain BCE. focal and class_balanced (Cui et al. 2019) losses are also available as options in the code.

Validation progression (macro-F1 @ 0.5): epoch 1: 0.690 → epoch 2: 0.697 → epoch 3: 0.732 (still rising; more epochs may improve it further).

---

Decision thresholds (severity-aware)

A fixed 0.5 threshold is not used. For each label, the threshold is chosen separately to maximize F_beta on validation. For high-severity categories (CSAE, SELF_HARM_SUICIDE, VIOLENT_CRIMES), beta=2 plus a recall floor (≥0.90) is applied — i.e. a miss (false negative) is prioritized over a false alarm.

Thresholds are stored in thresholds.json and applied at inference time.

---

Results

Test set (40,527 examples).

Overall

Per-category (test)

The low precision on VIOLENT_CRIMES and CSAE is intentional: the severity-aware threshold pins recall at ≥0.90 to minimize misses in these critical categories. Their AUPRC values (0.83–0.85) show the ranking quality is healthy and the low precision is only a threshold trade-off.

By source language

Native Turkish rows (social-media-sourced hate/harassment) are noisier and harder, so they score lower.

---

Usage

This is not a standard AutoModel; it is a custom two-head model. The repository includes the required modeling code (model.py, labels.py, predict.py, metrics.py), the weights (model.pt), the tokenizer, config.json, and thresholds.json.

Install

pip install torch transformers huggingface_hub

Self-contained inference

import sys, json, torch
from huggingface_hub import snapshot_download
from transformers import AutoTokenizer

repo = snapshot_download("thealper2/guardrail-tr-berturk")
sys.path.insert(0, repo)               # model.py, labels.py come from the repo
from model import GuardrailModel
from labels import CATEGORIES

cfg = json.load(open(f"{repo}/config.json"))
thr = json.load(open(f"{repo}/thresholds.json"))
tok = AutoTokenizer.from_pretrained(repo)
model = GuardrailModel.load(repo, cfg["base_model"], dropout=cfg["dropout"]).eval()

@torch.no_grad()
def guard(text: str):
    enc = tok(text, truncation=True, max_length=cfg["max_length"], return_tensors="pt")
    p_bin, p_multi = model.predict_proba(enc["input_ids"], enc["attention_mask"])
    p_bin = float(p_bin[0]); p_multi = p_multi[0].tolist()
    flagged = [c for i, c in enumerate(CATEGORIES) if p_multi[i] >= thr[c]["threshold"]]
    unsafe = p_bin >= thr["_BINARY"]["threshold"] or bool(flagged)
    return {
        "safety": "unsafe" if unsafe else "safe",
        "unsafe_score": round(p_bin, 4),
        "flagged_categories": flagged,
        "category_scores": {c: round(s, 4) for c, s in zip(CATEGORIES, p_multi)},
    }

print(guard("Birinin sosyal medya hesabını nasıl ele geçirebilirim?"))
# {'safety': 'unsafe', 'unsafe_score': 0.999,
#  'flagged_categories': ['NON_VIOLENT_CRIMES', 'PRIVACY_VIOLATION'], ...}

print(guard("Bugün hava nasıl, pikniğe çıkalım mı?"))
# {'safety': 'safe', 'unsafe_score': 0.0006, 'flagged_categories': [], ...}

As a filter in front of an LLM

def safe_chat(user_prompt, llm):
    verdict = guard(user_prompt)
    if verdict["safety"] == "unsafe":
        return f"Sorry, this request was blocked ({', '.join(verdict['flagged_categories'])})."
    return llm(user_prompt)

---

Limitations and risks

• Scope: The model classifies user prompts only; it is not designed to evaluate assistant responses.
• Translated/synthetic data: A large portion of the data is machine-translated and/or culturally adapted, and labels were produced by an LLM jury. Translation artifacts and label errors are possible.
• Weaker on native Turkish: Performance on tr-origin (social media) rows is lower than on en-origin rows.
• Rare classes: Precision is lower for low-support categories such as CSAE and MISINFORMATION_POLITICAL.
• Policy dependence: Category boundaries are design choices (especially harassment vs. hate, and political misinformation may be borderline).
• Out of scope: Labels are not legal, clinical, or political judgments and should not be used as such without human review.

---

License and citation

• Model license: Apache-2.0
• Dataset: ytu-ce-cosmos/guardrail-tr (Apache-2.0)
• Base model: dbmdz/bert-base-turkish-cased

@misc{guardrail_tr_berturk,
  title  = {Guardrail-TR: A BERTurk-based Turkish prompt safety classifier},
  note   = {Trained on the ytu-ce-cosmos/guardrail-tr dataset},
  year   = {2026}
}

The views expressed in the prompts do not reflect the views of the dataset curators or the model author.