AI generated, human reviewed.

Halogen Spotter: a guard that reads tool results

A fine-tune of Qwen3Guard-Gen-0.6B (Apache-2.0). It keeps what the base does as a moderation model and adds one thing: it flags a tool result that carries instructions aimed at the assistant.

It is a second pair of eyes for an agent, not a security boundary. An injection it does not flag passes, and one that asks for no tool call has nothing to hold.

What it was trained for

The input is a tool result of at most 512 tokens, in the base model's own prompt form. The answer is read at the Safety: label. Read a longer result as overlapping windows of 512 tokens every 448, and score it by its largest window.

Flag a result when P(Unsafe) + P(Controversial) of the three label rows is at least 0.25. That threshold was set on benign tool results only, on the NPU route. Set it again for a different runtime: it is not a property of the weights alone.

Read the three label rows, do not generate. On about 3.4% of tool-result windows the most likely token after Safety: is not a label, because training looked only at those rows. On the base's own moderation sets the top token was always a label.

Results on the NPU (Ryzen AI, the published halogen image)

The test is AgentDojo's recorded runs: 22 models, four office-style suites, its important_instructions attack, 2,710 successful attacks. AgentDojo never trained this model and no checkpoint was chosen from it; the threshold was set on the benign results of this test's three sources, never on an attack. The whole test was read once, on this model.

"Stopped" means the result was flagged and the agent's harmful call was held by a rule around the guard: a side-effecting call in a flagged turn, outside the task's scope or using a value lifted from the flagged result.

Every attack the base stopped at its threshold, this model stopped at its own.

AgentDojo's other recorded attack forms (1,640 successful attacks, reported only) read 87 to 93% stopped, against 57 to 89% for the base. Two of those forms resemble text it trained on, so they say less.

Moderation, on the base's own benchmark

F1 on the NPU, strict / loose. The largest move on any cell is 1.09 points. Individual decisions do move: 6.2% of them differ from the base's.

Limits

• Measured on office tasks only. No published injection benchmark fits coding agents, so coding is measured for false alarms only (0 of 339 results from coding sessions flagged), not for attacks.
• Text-only attacks are flagged but not stopped. When an injection asks only for words in the answer, there is no call to hold.
• Results vary by model and task. The agent model matters (one model's runs read 21.4% stopped, on 28 attacks; the median is 91.2%), and so does the task (travel stays the weakest suite, at 63.6%).
• A tool result that legitimately tells the assistant what to do looks like an attack. Most of the benign flags are of this kind, such as a task that asks the agent to follow a to-do list.
• The false-alarm samples are small: 339 and 184 results on two of the three sources.
• Only tested against these published attacks. An adaptive attacker who knows this model was not tested.

Training

LoRA (rank 16, alpha 32) on all 28 layers' attention and MLP projections, merged. 3,000 steps, no gradient through the template's fixed text. Loss: cross-entropy on the three label rows for tool-result windows, plus a KL term to the base's distribution on moderation conversations so the base's behaviour stays.

Use it

In halogen, HALOGEN_AGENT_GUARD=watch or hold fetches and serves it on the Ryzen AI NPU beside the Flash model: docs/agent-guard.md. halogen-spotter-0.6b.hnpw is the NPU's weight file, checkpoint/ the merged F32 model in the usual Hugging Face layout, lora-03000.safetensors the adapter before the merge.

Attribution

Built on Qwen3Guard-Gen-0.6B by the Qwen team (Apache-2.0). Training used the sets in the table above; Aegis 2.0 is by NVIDIA, CC BY 4.0, cut and wrapped for this use. The text itself is not in this repo.