Status: TriageKit v1.0 (triage only) — llama.cpp production path validated.
The supported deployment is two-file: soc-triage-base-Q4_K_M.gguf +
soc-triage-lora-F16.gguf + soc-triage-nonthink.jinja.
Validated on 15 golden test alerts: 93.3% label agreement (14/15), JSON and
contract validity 100%, 0 guardrail rejects. The one label difference is
unknown → suspicious (both route_human); two action differences are
route_human where the adapter auto-closed — the safer direction.
soc-triage-bolt-e4b-Q4_K_M.gguf: DEPRECATED (66.7%).mlx_lm.server only applies --adapter-path when the request model id is
default_model. Earlier server evaluations used the model path and therefore
measured the base model, not the adapter. Corrected golden macro-F1:
| Configuration | Macro-F1 | Coverage |
|---|---|---|
| Base Bolt + guardrails | 0.9541 | 0.309 |
| Pilot adapter | 0.3788 | 0.036 |
| Evidence adapter | 0.7828 | 0.182 |
Use the base model for quality; the adapter is not an improvement.
See evaluation/CORRECTION_2026-10-29.md. L1 Copilot kit: deployment/l1-copilot/.
detector/ contains small RandomForest detectors (OTRF + EVTX-MITRE features)
that run before the LLM in the gym bridge:
| Detector | EVTX test recall | OTRF acc | OTRF recall | OTRF FPR |
|---|---|---|---|---|
| OTRF-only | 24.6% | 96.0% | 100% | 14.3% |
| Combined | 100% | 92.0% | 94.4% | 14.3% |
Caveats: within-dataset (OTRF lab + attack-only EVTX); detector layer, not the
LLM; still needs real-SIEM validation. See detector/README.md.
These are real harness runs, not cherry-picked:
| Probe | Result |
|---|---|
| SOC-AI-Gym v0.5.0, OTRF real-lab triage (84 trials) | task success 0.214, FPR 0.0, ECE 0.575; 36/84 cases over context limit |
| SOC-AI-Gym demo fixtures (authored, 24 trials) | task success 0.125 |
| SOCBench (DeepTempo) tool-using agent, stratified smoke | verdict F1 0.667, precision 1.00 |
| Golden synthetic test | macro-F1 0.954 (synthetic only) |
Interpretation: the model is safe but conservative on real telemetry (abstains/benign, zero false positives) and is not yet competitive on GUIDE/OTRF-style triage. Labels/evidence typing and calibration are the next improvements (see harness repo docs).
| File | Size | Notes |
|---|---|---|
soc-triage-base-Q4_K_M.gguf |
5.0 GB | official BF16 base quantized to Q4_K_M; production base |
soc-triage-lora-F16.gguf |
3.3 MB | GGUF LoRA adapter; load with --lora |
soc-triage-nonthink.jinja |
1 KB | required chat template (thinking disabled) |
soc-triage-bolt-e4b-Q4_K_M.gguf |
4.9 GB | DEPRECATED merged single-file (66.7% agreement); kept for reference |
The officially published base GGUF (ggml-org/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF)
can be combined with the small LoRA file:
llama-server -m NVIDIA-Nemotron-3.5-Bolt-E4B-Q8_0.gguf \
--lora soc-triage-lora-F16.gguf --jinja -c 4096 --port 8093
Validated 2026-10-26 on 15 golden test alerts with the non-thinking template
(soc-triage-nonthink.jinja):
| Metric | Result |
|---|---|
| JSON validity | 100% |
| Contract validity | 100% |
| Guardrail rejects | 0 |
| Label agreement vs MLX adapter | 86.7% (13/15) |
| Action agreement | 93.3% |
| Differences | 2 × unknown → suspicious (both route_human; safe direction) |
Production path (validated 2026-10-26): base Q4_K_M + LoRA → 93.3% label
agreement (14/15), JSON/contract 100%, 0 rejects. The single label difference
is unknown → suspicious (both route_human); the two action differences are
route_human where the adapter auto-closed — the safer direction. This meets
the 90% regression gate.
Recommended llama.cpp serving (uses the bundled 5 GB base):
llama-server -m soc-triage-base-Q4_K_M.gguf \
--lora soc-triage-lora-F16.gguf \
--chat-template-file soc-triage-nonthink.jinja \
--jinja -c 4096 --port 8093
For reference, the Q8/BF16 base variants were also tested (86.7% each): the
MLX adapter is trained/served on the 4-bit base, so the Q4_K_M base is the
correct match.
```bash
curl -L -o NVIDIA-Nemotron-3.5-Bolt-E4B-Q8_0.gguf \
https://huggingface.co/ggml-org/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF/resolve/main/NVIDIA-Nemotron-3.5-Bolt-E4B-Q8_0.gguf
llama-server -m NVIDIA-Nemotron-3.5-Bolt-E4B-Q8_0.gguf \
--lora soc-triage-lora-F16.gguf \
--chat-template-file soc-triage-nonthink.jinja \
--jinja -c 4096 --port 8093
SHA256SUMS covers the model and the audit artifacts in config/, evaluation/,
training/, deployment/.
| Check | Result |
|---|---|
| Harness FP gates (attack-flag / escalation / unsafe close) | 0 / 0 / 0 |
| Live Wazuh routine control (59 alerts) | 0 flagged |
| Live Wazuh attack control (48 alerts) | 0 downgraded |
| SOCBench agent stratified smoke (8 units × 4 personas) | precision 1.00, 0 false-malicious, F1 0.667 |
| Artifact | JSON valid | Contract valid | Label agreement vs adapter |
|---|---|---|---|
| Q4_K_M | 100% | 100% | 66.7% |
| Q6_K | 100% | 100% | 46.7% |
| Q8_0 | 100% | 100% | 46.7% |
All disagreements were guardrail repairs to unknown because the GGUF cited
the wrong evidence ID (e.g. the baseline item instead of the config audit).
Safe (no auto-close) but below the 90% regression gate. Rendering the exact
training chat template (instead of relying on the embedded template) restores
the correct labels in a 5-case check, but evidence-ID grounding still slips;
a retrain with semantic evidence IDs is the proper fix.
Conversion chain: MLX 4-bit base + LoRA adapter → fused & dequantized to BF16 →
GGUF F16 (convert_hf_to_gguf.py, llama.cpp) → Q4_K_M (llama-quantize).
Only the language tower is included (no vision/audio projector).
llama-server -m soc-triage-bolt-e4b-Q4_K_M.gguf --jinja -c 4096 --port 8093
Then call the OpenAI-compatible endpoint with the harness system prompt. The production system prompt requests English rationales and a JSON decision:
{"decision": "<label>", "confidence": 0.0, "evidence_refs": ["EV-1"], "rationale": "...", "next_step": "..."}
Notes:
temperature=0 and cap output at ~200 tokens.Same harness numbers as the adapter: macro-F1 0.954, missed malicious 0, auto-close precision 1.0, attack-flag FP 0; gates READY for L0 shadow (11 PASS + 3 SKIP under the statistical guard). See the adapter card for the full table; these numbers apply to synthetic data only.
tegridydev/opensec-triage — CC BY 4.0.