safespec-probe-qwen3guard-stream-0.6b
An agent-hijack / prompt-injection detector built as a LoRA-fine-tuned variant of Qwen3Guard-Stream-0.6B, plus a tiny binary classification head on top of the last-token hidden state.
This model is the safety probe used by SafeSpec, a reference implementation of asynchronous safety probing for speculative decoding. In the SafeSpec setup the probe is injected into the draft/verify loop of a Qwen3-0.6B → Qwen3-8B speculative decoder and contributes streaming hijack detection with less than 3 % wall-time overhead on a single GPU.
Headline numbers
Evaluated on 101 out-of-distribution test samples from the AgentLAB benchmark (23 held-out environments never seen during training).
| metric | value |
|---|---|
| Test AUC | 0.961 |
| Test FNR @ τ=0.9 | 2.2 % (1 / 46) |
| Test FPR @ τ=0.9 | 5.5 % (3 / 55) |
| Streaming FNR (in SafeSpec, single-GPU) | 2.2 % |
| Streaming FPR (in SafeSpec, single-GPU) | 5.6 % |
| Wall-time overhead vs no-probe baseline | +2.19 % (mean) / +1.58 % (median) |
| p10 wall-time delta | −1.68 % (10 % of samples faster with probe) |
| Acceptance-rate impact on speculative decoding | −0.11 % (noise-level) |
The 1 false negative on the overall test split is a judge-labelling artefact (benign chemistry QA mis-labelled as hijack by the dataset's LLM-as-judge). On the real agentic attack subset (11 positives) the probe achieves 11 / 11 = 100 % detection.
Full numbers and ablations: see eval/metrics.json
and the SafeSpec experiment doc.
Files in this repository
├── README.md this file
├── model.safetensors merged base + LoRA weights (ready to load)
├── config.json model config
├── configuration_qwen3.py trust_remote_code: config class
├── modeling_qwen3_guard.py trust_remote_code: model class
├── generation_config.json
├── tokenizer.json, tokenizer_config.json, special_tokens_map.json
├── vocab.json, merges.txt, added_tokens.json, chat_template.jinja
├── hijack_head.pt Linear(1024, 1) + sigmoid head (4 KB)
├── merge_lora.py script to re-produce model.safetensors from lora/
├── lora/ LoRA-only adapter (for stacking on fresh bases)
│ ├── adapter_config.json
│ ├── adapter_model.safetensors (17 MB)
│ └── hijack_head.pt
└── eval/
├── metrics.json key AUC / FNR / FPR / τ / timing
├── training.json per-epoch training log
└── test_predictions.json per-sample output probabilities
- Use the merged
model.safetensorsfor drop-in deployment (one-shotAutoModel.from_pretrained(...)). - Use the
lora/adapter if you want to stack this LoRA on a different (freshly fine-tuned) Qwen3Guard-Stream-0.6B base.
How to use
Option A — load the merged model (zero config, recommended)
After huggingface-cli download husj576/safespec-probe-qwen3guard-stream-0.6b --local-dir ./probe_ckpt, everything you need lives in ./probe_ckpt/:
the LoRA adapter is already fused into model.safetensors, so you only
need to load the base and attach the small binary head.
import torch
import torch.nn as nn
from transformers import AutoModel, AutoTokenizer
repo = "husj576/safespec-probe-qwen3guard-stream-0.6b"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(
repo, torch_dtype=torch.bfloat16, trust_remote_code=True
).cuda().eval()
# Load the binary hijack head (4 KB, Linear(hidden_size, 1) + sigmoid).
# When downloading via huggingface_hub, hijack_head.pt lands in the same
# local directory as model.safetensors.
from huggingface_hub import hf_hub_download
head_path = hf_hub_download(repo_id=repo, filename="hijack_head.pt")
hidden_size = model.config.hidden_size
hijack_head = nn.Linear(hidden_size, 1, dtype=torch.bfloat16).cuda().eval()
hijack_head.load_state_dict(torch.load(
head_path, map_location="cuda", weights_only=True,
))
def score(messages):
text = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=False
)
ids = tokenizer(text, return_tensors="pt").input_ids.cuda()
with torch.inference_mode():
out = model(ids, output_hidden_states=True)
last = out.hidden_states[-1][:, -1, :] # (1, hidden)
return torch.sigmoid(hijack_head(last)).item()
prob = score([
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of Iceland?"},
{"role": "assistant",
"content": "Reykjavik is the capital of Iceland."},
])
print(f"hijack probability = {prob:.3f}")
Option B — load base + LoRA adapter (keeps original weights separate)
from peft import PeftModel
base_repo = "Qwen/Qwen3Guard-Stream-0.6B"
lora_repo = "husj576/safespec-probe-qwen3guard-stream-0.6b"
base = AutoModel.from_pretrained(base_repo, torch_dtype=torch.bfloat16,
trust_remote_code=True).cuda().eval()
# Qwen3Guard-Stream exposes the underlying LM as `base.model`
base.model = PeftModel.from_pretrained(base.model, f"{lora_repo}/lora")
# ... then follow the same `score()` pattern as Option A, loading
# hijack_head.pt from `lora/hijack_head.pt`.
Option C — plug into SafeSpec speculative decoding
from safespec import SpecDecoderWithProbe, BatchedGuardProbe
# Pass the merged checkpoint dir -- BatchedGuardProbe auto-detects
# hijack_head.pt sitting next to model.safetensors.
probe = BatchedGuardProbe(
model_path="./safespec-probe-qwen3guard-stream-0.6b",
)
engine = SpecDecoderWithProbe(
probe=probe,
async_probe=True,
probe_batch_rounds=6,
defer_final_flush=True,
)
# ... see https://github.com/hsj576/SafeSpec/blob/main/examples/quickstart.py
Training details
| hyper-parameter | value |
|---|---|
| Base model | Qwen3Guard-Stream-0.6B (frozen) |
LoRA rank r |
16 |
LoRA alpha |
32 |
| LoRA dropout | 0.05 |
| LoRA target modules | q_proj, k_proj, v_proj, o_proj (every transformer block) |
| Hijack head | Linear(1024, 1) + sigmoid |
| Trainable parameters | 4.588 M (0.76 % of backbone + 1025 head params) |
| Training data | SafeSpec phase-3 (225 training rows, 42 dev, 101 test; env-split) |
| Objective | binary cross-entropy (pos_weight = 1.0) |
| Optimiser | AdamW, lr = 1e-4, grad_accum = 8 |
| Epochs | 3 |
| Precision | bfloat16 |
| Wall-clock | ~50 s epoch-0, ~27 s epoch-{1,2} on one H800/A100 |
Input format during both training and inference:
<|im_start|>system
{system prompt}<|im_end|>
<|im_start|>user
{user prompt (may include tool observation / payload)}<|im_end|>
<|im_start|>assistant
{full assistant output}<|im_end|>
The hijack head reads the last-token hidden state of this whole
sequence. In streaming use this means the authoritative score is produced
at _finalize time; mid-stream probabilities are noisier and should not
be thresholded directly (see the "Known limitations" section below).
Intended use
Direct use
- Post-hoc classification of agentic conversation transcripts for prompt-injection / hijack events.
- Streaming observer inside a speculative decoder (the SafeSpec reference setup), with the last-token score read at end-of-stream.
Downstream use
- Dataset filtering: flag suspicious tool outputs or assistant replies in logs.
- Re-training on your own domain data: start from this LoRA + head as an initialisation and fine-tune further.
Out-of-scope
- Real-time mid-stream interception at arbitrary token positions — the head was trained on full-sequence last-token hidden states, so mid-stream probabilities are not reliable (see Known limitations).
- Generative safety filtering of user inputs (toxicity, harmful instructions) — for that use case prefer the native Qwen3Guard-Stream-0.6B head without this LoRA.
Known limitations
- Mid-stream probabilities are noisy. Use
asst_last_lora_hijack_prob(set at_finalizetime) rather than the running maximum. A streaming-aware head is on the roadmap. - FPR is measured on a mixed subset. The 5.5 % test FPR includes non-agentic QA samples (e.g. "What is the capital of Iceland?") drawn from a refused-set. On a pure agentic-benign benchmark (BFCL-v4) the authors expect FPR to drop further; evaluation is in progress.
- Training data is small (225 rows). The reported OOD generalisation is strong (test from 23 unseen environments) but it is a 0.6 B probe trained on a small set; verify on your domain before production use.
- Only validated against Qwen3-8B outputs. Cross-model transfer (e.g. Llama-3.1 agents) is an open question.
Ethical considerations
The model is intended as a defensive safety probe. It is not a toxicity or harm classifier, and it should not be used to make high-stakes decisions without human-in-the-loop review. The training data contains synthetic adversarial payloads; false positives on benign content are possible (see Known limitations §2).
Citation
@software{safespec2026,
title = {SafeSpec: Asynchronous Safety Probing for Speculative Decoding},
author = {Shijing Hu},
year = {2026},
url = {https://github.com/hsj576/SafeSpec},
license = {Apache-2.0}
}
Base model:
@misc{qwen3guard2025,
title = {Qwen3Guard},
author = {Qwen Team},
year = {2025},
url = {https://huggingface.co/Qwen/Qwen3Guard-Stream-0.6B}
}
License
Apache-2.0. The base model (Qwen3Guard-Stream-0.6B) is released by the Qwen team under its own license; please refer to the base model's model card for its terms.