Decision-WAF-V1-0.1B

A compact request classifier for a web application firewall. It reads a single HTTP request — method, path, query, and headers inspection; bodies are not inspected and answers one question: is this request benign, or which attack class is it? It replies with a single letter token, which the decision-waf ingress scores as a probability distribution over the categories. It is the smallest member of the Decision-WAF family: 149M parameters on the AxiomicLabs/GPT-X3-150M base, a hybrid of 18 GDN2 linear-attention layers and 6 GQA layers with RoPE.

What it is designed for

• Per-request WAF verdicts: block-or-allow with a tunable threshold on the attack score (1 − p(benign)).
• HTTP inspection: classify requests using the method, path, query, and headers without inspecting request bodies.
• Defense in depth, not sole protection: it is one signal in front of real services, alongside TLS, authentication, and backups. It is not an exploit detector or a substitute for patching.

Categories

benign · auth_abuse · file_access · injection · other_attack · recon · ssrf_redirect · xss

Intended use and serving

• Loads via transformers with trust_remote_code=True (the custom TX5 modeling files ship in this repository); fine-tuning requires flash-linear-attention>=0.6.0 on CUDA for the GDN2 fast path.
• No GGUF yet: the TX5 architecture is not in llama.cpp. Serve via transformers or vLLM behind a systemone-compatible adapter, or use the larger family members for llama.cpp-native serving.
• Latency: ~2.5 s per request on an Apple M-series GPU (batch 1, MPS, unoptimized reference path); sub-100 ms class expected on CUDA with fla kernels.

Training

LoRA r=64, α=128, approximately 15M trainable parameters on GPT-X3-150M, merged into the base weights. One-token classification head readout; loss on the answer token only. Trained for 4,591 steps (approximately 2.2 h) on a single RTX 4000 SFF Ada, using exact-length batching so the GDN2 layers run their flash-linear-attention kernels.

Data: approximately 150k labeled HTTP requests, with the evaluation set strictly held out via request-line, query-value, and header-value fingerprints. Same data recipe as Decision-WAF-V1.1-0.8B.

Evaluation

The benchmark set (299 requests: 145 benign, 154 attacks) uses the same harness for all models. h2o-lightning-4b and d1-3B numbers are from the official decision-waf-eval reports, jev-latest from api.typesafe.ai (threshold grid 0.05–0.95); Decision-WAF models are computed from per-row scores on the same rows, backend and scoring semantics (attack score = 1 − p(benign)).

Threshold sweep (attack recall / false-positive rate):

Best F1 per model (dots on the chart):

Known limitations:

• Outperformed by every other Decision-WAF model and both reference models on this benchmark; use a larger family member for production traffic screening.
• Does not detect header-borne Log4Shell/Shellshock payloads (scores ~0.0001; the 0.4B and 0.8B models score 0.999+).
• High validation accuracy (0.983) against far lower benchmark recall indicates the model fits the training distribution but generalizes weakly — a capacity limit at this scale, not a data gap.
• Headers-only: body-borne attacks are out of scope in the current inspection mode.
• Trained on curated and synthetic data; distribution shift can move both error directions. Monitor and retrain.
• A WAF classifier can be evaded and can misclassify; use as one layer, keep audit logs, and never let it be the only thing between the internet and a vulnerable service.

Files

• model.safetensors + tokenizer — merged HF weights, with the custom TX5 modeling files (modeling_gptx3.py, configuration_gptx3.py) required by trust_remote_code.
• No GGUF: TX5 is not yet a llama.cpp architecture.