SpanGuard suite — every model, eval record, chart and fixture in one organized repo
The complete, ready-to-verify package behind SpanGuard v5.0.0 (release v5.0.0 · live showcase · demo Space).
Repo map
models/ the four CPU-tier guards (each ~5 MB, torch, no GPU needed)
spanguard-char/ char spans + doc verdict, corpus-v1/v2 tier (0.79M)
spanguard-char-v3/ + MCP / skill / buried / md-image / Bangla (0.79M)
spanguard-char-v4/ + log / code / rag / Arabic / Japanese (812k)
spanguard-fm-1/ SPG-FM-1: one model for every domain (v5) (1.70M)
eval/ committed run records (release runs per version) + the
benchmark fixtures they were measured on (JSONL, fictional)
charts/ the measured v5 charts + the SVG benchmark set
corpus/ corpus manifests — seed-lock proof for the training data
code/ spanguard-head.zip — the full source snapshot (Apache-2.0)
The 135M models live in their own repos: spanguard-span-mdistil-v2 (span backbone) · docguard · toolguard.
Measured results (release run eval/v5-ship-1548.run.json)
| set | SPG-FM-1 (1.7M) | fast-v4 (rules + 135M span + 3-level router) |
|---|---|---|
| hand-written OOD · 62 docs | 10.4% escape · 26.4% over-strip | 4.2% escape · 17.5% |
| agent traffic · 900 docs | 0.0% escape · 5.2% | 0.0% escape · 5.1% |
| public probe · 452 docs (over-strip only) | 0.0% · 45.4% | 0.0% · 24.6% |
| extended attacks · 88 docs · 5 families | 0.0%/0.0% · doc F1 1.000 · span F1 0.917 | 0.0%/0.0% · doc F1 1.000 |
| extended-v4 · 108 docs · 7 languages | 0.0%/0.0% · doc F1 1.000 · span F1 0.901 | 0.0%/0.1% · doc F1 0.993 |
| latency (2-thread CPU) | p50 36.7 ms | p50 14.6 ms |
Per-language doc escape on extended-v4 — SPG-FM-1: en 0.0 · de 0.0 · hu 0.0 · fr 0.0 · bn 0.0 · ar 0.0 · ja 0.0 (one forward pass, no router, no language detection).
Usage
import os
from huggingface_hub import snapshot_download
os.environ["SPANGUARD_FM_MODEL"] = snapshot_download("Nabidnur/spanguard-suite")
# ^ or point at models/spanguard-fm-1 inside a local clone of this repo
from spanguard.detect.pipeline import Guard
r = Guard(mode="fm").guard(tool_output_text)
r.cleaned # payload spans replaced with typed [stripped:*] markers
r.attacked # document-level verdict
r.stripped # findings with offsets + payload hashes
Install: pip install "spanguard @ git+https://github.com/Hisernberg/spanguard.git"
(+ CPU torch). Other tiers: Guard(mode="rules" | "fast" | "fast-v2" | "fast-v3" | "fast-v4")
with SPANGUARD_CHAR_MODEL / _V3 / _V4 pointing at the matching models/ dirs.
Re-verifying the numbers
pip install "spanguard @ git+https://github.com/Hisernberg/spanguard.git"
export SPANGUARD_FM_MODEL=<downloaded models/spanguard-fm-1>
export SPANGUARD_CHAR_MODEL=<models/spanguard-char> SPANGUARD_CHAR_MODEL_V3=<models/spanguard-char-v3>
export SPANGUARD_CHAR_MODEL_V4=<models/spanguard-char-v4>
python3 -m spanguard.eval.cli --detectors spg-fm,spanguard-fast-v4 \
--sets handwritten,agent-traffic,public-deepset,extended-attacks,extended-v4 --tag my-run
The fixtures this compares against ship in eval/ — escape and over-strip are always
reported together; a guard that deletes everything escapes 0% and is useless.
Honesty
- Trained and benchmarked on fictional data only (deterministic seed-locked generator;
manifests in
corpus/prove the seed lock; no personal data anywhere). - The known trades are published: fm over-strips out-of-domain benign prose (45.4% / 26.4%);
the router stack keeps fidelity (24.6% / 17.5%). The v5 orchestration repair and its
broken intermediates are committed and auditable in the repo's
eval/results/. - Out of scope: jailbreaks typed by the app's own user, malware analysis, DLP, and any safety guarantee. Stripping is a mitigation, not proof of safety.
- Self-rating: 8.3/10 rubric.