PolicyGuard 🧭

PolicyGuard is a lightweight guardrail model trained for policy-trajectory violation detection in autonomous web agents.
It identifies whether an agent’s long-horizon trajectory complies with externally imposed or human-specified policies.
Built on the PolicyGuardBench dataset, it achieves high accuracy, cross-domain generalization, and remarkable efficiency — proving that compact guardrails can be both accurate and deployable.

For more details, refer to our paper: Towards Policy-Compliant Agents: Learning Efficient Guardrails For Policy Violation Detection.

🧩 Key Objectives

1. Detect policy violations in agent trajectories (beyond single-turn safety).
2. Generalize across domains (e.g., shopping, GitLab, Reddit, map, admin).
3. Enable prefix-based early detection — anticipating violations before completion.
4. Optimize for inference efficiency with a 4B-parameter model.

---

⚙️ Model Overview

• Model name: PolicyGuard-4B
• Base model: Qwen3-4B-Instruct-2507
• Training objective: Binary classification of violation / no-violation
• Dataset: PolicyGuardBench — 60k policy–trajectory pairs
• License: apache-2.0
• Input format: concatenated policy + trajectory description
• Output format: “violation” / “no violation”

---

📊 Benchmark Results

➡️ PolicyGuard-4B achieves near-frontier accuracy at <1/10 inference cost, demonstrating that efficient, small-scale guardrails are feasible:contentReference[oaicite:1]{index=1}.

🧰 How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch, re

model_id = "Rakancorle1/PolicyGuard-4B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

def classify(policy, actions):
    if isinstance(actions, list):
        actions = "\n".join(actions)
    prompt = f"Policy: {policy}\n\nTrajectory Actions:\n{actions}\n\nOutput violation or no_violation."
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    outputs = model.generate(**inputs, max_new_tokens=4)
    text = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True).lower()
    return "no_violation" if "no_violation" in text else "violation"

# Example
policy = "Do not submit a form without filling mandatory fields."
actions = ["Open form page", "Click submit without input"]
print(classify(policy, actions))
# -> 'violation'