🌟 Sepal-0.8B

A typed decision model for security operations (SOC) work

NOTE

• This is the full training run of Sepal-0.8B. It replaces the earlier test run on this repo (tag full08b-test-run, still available).
• The scores below are from our own development panels. Calibration on the held-out report half (ECE 0.031) is just above our 0.030 target.
• Larger sizes (2B, 4B, 9B) on the same recipe will follow.

📋 Overview

• Sepal answers typed questions about a given state (an alert, a log window, a policy, a document) in one pass: yes/no (noul), choice among up to 255 options, and score levels. Every answer comes with calibrated probabilities, not free text.
• It is specialised for SOC decisions: alert verdicts, rule and policy matching, ATT&CK mapping, vulnerability priority, injection detection in attacker-controlled fields, response gating, and "insufficient evidence" abstention, with general decision data mixed in to keep broad skills.
• New in this version: long log windows (states up to about 16k tokens) with many custom yes/no criteria, and tool and intent selection.
• It is an adapter (LoRA rank 64 plus a decision head) on the post-trained Qwen3.5-0.8B.

🔧 Method

• Trained with Unsloth's decision trainer (Clef head, one prompt per record) using LoRA rank 64, alpha 128, on every linear layer of the language model; lr 1e-4, cosine with 10% warm-up, one epoch, 64 records a step.
• The recipe combines the three levers that won our screening round: the post-trained Qwen3.5-0.8B as the starting point, a tool and intent family, and long log windows up to true 16k states.
• Training corpus full-v15: 691,705 records, 449.4M tokens, 149 sources (security layer about 41% of tokens), including generated SOC data, long log windows with custom criteria, tool-selection data built from training splits only, public security datasets, and general decision datasets. Every training record was screened against our benchmark: zero exact or near copies, and no shared instances.
• 10,808 steps, 11.5 GPU-hours on one RTX PRO 6000 (96 GB), training inputs up to 24,576 tokens.
• One temperature per question type and option-count bucket, fitted on held-out datasets, is stored in sepal.json.

📊 Scores

Development panels of our own benchmark (SOCDecide 0.7.0), our long-log and tool panels, and held-out general datasets. Test partitions were not read.

Public Decision Index 0.3 (our own run, HLE not included): coming soon. The earlier test run scored 16.63.

📁 Files

• adapter_model.safetensors, adapter_config.json: the LoRA adapter for Qwen/Qwen3.5-0.8B (revision 2fc06364715b967f1860aea9cf38778875588b17).
• joint_head.safetensors, joint_head_config.json, unsloth_decision_config.json: the decision head read by Unsloth's decision trainer.
• sepal.json: training settings, versions and the fitted temperatures.
• Tokenizer files from the base model.

💫 Quantization

Coming Soon!

🤝 Contributors

Developed with ❤️ by BlossomAI

---

_{Star ⭐️ this repo if you find it valuable!}