A LoRA adapter and decision head that turn inclusionAI/Ling-3.0-tiny into a safety gate for agent trajectories. Given a user request, the agent's tool calls, observations and final reply, it returns one calibrated probability that the trajectory is unsafe. Two thresholds turn that probability into allow, ask or deny.
Trained with AReno's JEV decision head (commit e17794c). Full recipe, code and evaluation: ling-cookbook, Agent Safety Gate.
| File | Content |
|---|---|
adapter_config.json, adapter_model.safetensors |
LoRA rank 8, alpha 16, on q/k/v/o and the expert and shared-expert MLPs (PEFT format) |
score_head.safetensors, jev_config.json |
Linear-GELU-Linear decision head on the last token's hidden state |
frozen_thresholds_cpu.json |
temperature, decision threshold and ask tier, fitted on CPU dev scores |
The files are exactly the ones that were evaluated. adapter_config.json still names the training machine's path as the base model; point --model-path at your copy of Ling-3.0-tiny.
areno serve --model-path /path/to/Ling-3.0-tiny --decision-checkpoint /path/to/this/repo --host 127.0.0.1 --port 8103
POST /v1/systemone with {"state": "<trajectory text>", "questions": {"unsafe": <question>}} returns answers.unsafe.noul = P(unsafe). The trajectory format, the question and a client that applies the thresholds (gate_client.py) are in the recipe.
Test slices from task families no model trained on (2,154 records). Scored once, on CPU, with thresholds frozen on CPU dev scores.
| Model | Pre-Ex-Bench | ASSE-Security | ASSE-Safety | R-Judge | Mean |
|---|---|---|---|---|---|
| Majority class | 0.596 | 0.574 | 0.533 | 0.517 | 0.555 |
| Ling-3.0-tiny base | 0.632 | 0.579 | 0.612 | 0.543 | 0.592 |
| Safiron | 0.935 | 0.620 | 0.451 | 0.389 | 0.599 |
| ShieldAgent | 0.509 | 0.715 | 0.810 | 0.812 | 0.711 |
| This adapter | 0.733 | 0.983 | 0.820 | 0.674 | 0.802 |
Accuracy. F1, MCC, AUROC and the allow / ask / deny rates are in the recipe.
LICENSE). The base model, inclusionAI/Ling-3.0-tiny, is also MIT-licensed.@article{luo2025agentauditor,
title={AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents},
author={Hanjun Luo and Shenyu Dai and Chiming Ni and Xinfeng Li and Guibin Zhang and Kun Wang and Tongliang Liu and Hanan Salam},
journal={arXiv preprint arXiv:2506.00641},
year={2025}
}
adapter_model.safetensors 595 MBscore_head.safetensors 9 MBinclusionAI/Ling-3.0-tiny