Decision-WAF-V1-0.4B

A compact request classifier for a web application firewall. It reads a single HTTP request — method, path, query, and headers inspection; bodies are not inspected — and answers one question: is this request benign, or which attack class is it? It replies with per-option probabilities, which the decision-waf ingress scores as a verdict with a tunable threshold on the attack score (1 − p(benign)).

Unlike the 0.8B models in this family (decoder LMs answering with a single letter token), V1-0.4B is an encoder decision model: a ModernBERT-large backbone (421M) fine-tuned from convaiinnovations/laya (typed-decisions checkpoint), scoring each category option at masked marker positions in one forward pass.

What it is designed for

• Per-request WAF verdicts at the edge: block-or-allow in one forward pass, with a tunable threshold on the attack score (1 − p(benign)).
• HTTP inspection: classify requests using the method, path, query, and headers without inspecting request bodies.
• Defense in depth, not sole protection: it is one signal in front of real services, alongside TLS, authentication, and backups. It is not an exploit detector or a substitute for patching.

Categories

benign · auth_abuse · file_access · injection · other_attack · recon · ssrf_redirect · xss

Intended use and serving

• Served via llama.cpp (llama-server with laya decision-model support) through the /v1/systemone endpoint, or the laya SDK (laya-serve, Jev-compatible both ways). GGUF requires -b 4096 -ub 4096 (or larger): the whole request must fit one micro-batch.
• Recommended threshold: 0.5 (see evaluation). Scores are sharply bimodal and calibrated; the recall/FPR trade is flat from 0.05 to 0.8.
• Latency: ~0.35 s per request on an Apple M-series GPU (batch 1, Metal); Q8_0 GGUF is metric-identical to the full-precision checkpoint on the benchmark set.

Training

Full fine-tune of the laya typed-decisions checkpoint (ModernBERT-large, 421M total) via RLCD — REINFORCE with a group-mean baseline against proper scoring rules — with calibration temperatures fitted on a held-out calibration slice. 4 epochs / 9,100 optimizer updates (approximately 19 h) on a single RTX 4000 SFF Ada.

Data: approximately 150k labeled HTTP requests, with the evaluation set strictly held out via request-line, query-value, and header-value fingerprints.

Evaluation

The benchmark set (299 requests: 145 benign, 154 attacks) uses the same harness for all models. h2o-lightning-4b and d1-3B numbers are from the official decision-waf-eval reports, jev-latest from api.typesafe.ai (threshold grid 0.05–0.95); Decision-WAF models are computed from per-row scores on the same rows, backend and scoring semantics (attack score = 1 − p(benign)).

Threshold sweep (attack recall / false-positive rate):

Best F1 per model (dots on the chart):

It catches header-borne Log4Shell/Shellshock payloads (0.999) and correctly handles benign requests with non-browser clients and well-known service URLs in parameters; on the WAF-Bench confuser set it detects 56/57 attacks including all 14 evasion cases.

Known limitations:

• High-confidence false positives on benign search terms that look like attacks (1 or 2 players, cat /etc, ${budget}): score 0.997+, so no threshold separates them. A follow-up fine-tune upweighting that lookalike family is the identified fix.
• Headers-only: body-borne attacks are out of scope in the current inspection mode.
• Trained on curated and synthetic data; distribution shift (new apps, unusual API shapes, encodings) can move both error directions. Monitor and retrain.
• A WAF classifier can be evaded and can misclassify; use as one layer, keep audit logs, and never let it be the only thing between the internet and a vulnerable service.

Files

• model.safetensors, encoder/, tokenizer/, questions.json, rl_agent_config.json — the complete fine-tuned laya checkpoint (loadable with laya.load(<dir>)).