Cribl Decision 2

A small, fast model that answers multiple-choice questions about IT, security and telemetry data: which log type is this, is this event suspicious, which field holds the user, how severe is this alert. You give the model a state (any text: log lines, an inventory, a config, a document excerpt), a question and a list of options; it returns a calibrated probability for every option in a single forward pass, with no text generation.

4.2B parameters (Qwen3.5-4B text backbone + option scorer). For a smaller, faster model, see Cribl Decision 2 Mini (0.8B).

Quick start

pip install -r requirements.txt   # torch, transformers>=5.17, safetensors, huggingface_hub
hf download cribl-ai/cribl-decision-2 decision.py example.py requirements.txt --local-dir cribl-decision-2
cd cribl-decision-2 && python example.py

from decision import DecisionModel, Question

model = DecisionModel.from_pretrained()      # downloads this repo (8.4 GB); or pass a local folder

result = model.decide(
    state="Oct 27 14:02:11 fw01 kernel: [UFW BLOCK] IN=eth0 SRC=203.0.113.7 DST=10.0.0.5 PROTO=TCP DPT=22",
    question="Which log type is this?",
    options={"Linux firewall (netfilter/UFW) log": None, "OpenSSH server log": None, "Other / custom log type": None},
)
result.answer          # 'Linux firewall (netfilter/UFW) log'
result.probabilities   # {'Linux firewall (netfilter/UFW) log': 1.0, 'OpenSSH server log': 0.0, ...}

# Many questions at once (each is its own sequence; batches are padded on the right)
model.decide_batch([
    Question(state="user=alice action=login result=failure count=37 window=60s",
             question="Is this likely a brute-force attempt?", kind="boolean"),
    Question(state="CPU 98% for 45 minutes on db-03; p99 latency 4.2 s", question="How severe is this?",
             options=["informational", "low", "medium", "high", "critical"], kind="score"),
], batch_size=8)

With transformers only. The same model loads through AutoModel with remote code (this runs decision.py from this repository):

from transformers import AutoModel

model = AutoModel.from_pretrained("cribl-ai/cribl-decision-2", trust_remote_code=True)   # a DecisionModel
model.decide(state="...", question="...", options=["...", "..."])

Without trust_remote_code=True, AutoModel loads only the backbone, which cannot score options on its own. Text-classification pipeline()s are not supported.

Question kinds. choice (default): options is a list of labels or a mapping label → description (None for no description); descriptions are shown to the model as label — description. boolean: the answer is "false" or "true" (optionally pass {"false": ..., "true": ...} descriptions). score: options are ordered levels, lowest first; besides the distribution, the result carries expected, the probability-weighted level number (1 = lowest), e.g. low/medium/high at 0.05/0.30/0.65 gives 2.60.

Hardware. Runs on CUDA, Apple Silicon (MPS) or CPU in bf16. On CUDA, installing flash-linear-attention and causal-conv1d makes the linear-attention layers much faster. Prompts are limited to 4,096 tokens; longer inputs raise an error rather than being truncated.

Files. The repository is a standard transformers checkpoint of the Qwen3.5 text backbone plus decision_head.safetensors (the option scorer's query and key projections and one reverse_gate.<layer> scalar per reverse-reading layer) and decision_config.json (prompt template, special tokens, reverse-reading layers). decision.py (about 200 lines) is the reference for how they fit together. The prompt is the tokenizer's chat template exactly as used in training, including the assistant header and empty think block after the readout token; the answer is read at the readout token.

Model

The prompt lists the question and its options first and the input last, and ends with a readout token. Three changes to the Qwen3.5 backbone turn it into a reader that scores options in one pass:

• Bidirectional full attention. The 8 full-attention layers (of 32) see the whole prompt in both directions.
• Reverse reading. The linear-attention layer just before each full-attention layer also reads the prompt right to left with the same weights; the two directions are mixed through a learned gate. The other linear-attention layers read left to right as in the base model.
• Option scorer. Every option ends with its own marker token. An option's logit is a low-rank (256) bilinear score between the final hidden state at the readout token and the one at that option's marker; a softmax over the question's options gives the probabilities. There is no table of answer labels, so the number of options is limited only by the prompt length (training used up to 255).

Checkpoint. The released weights are the equal-weight average of two fine-tuning runs with different seeds. Averaging two separately trained scorers shrinks their logits, so the average carries one temperature (0.50) fitted on held-out general dev questions, never on the benchmarks below, and folded into the scorer's weights.

Training

Training ran in three stages, starting from Qwen3.5-4B:

1. General stage. ~1M multiple-choice and classification questions converted from public NLP datasets.
2. Reader stage. ~130k general questions in the final prompt format, to train the reverse-reading layers and the new option scorer.
3. Domain stage. ~20k questions: about half on IT, security and telemetry, and half general questions replayed from stage 1 to keep the model's general ability.

Evaluation

Telemetry Decision Bench v1.0 (Cribl, internal): 4,806 questions over 39 task types in ten areas: onboarding a new source, normalising and enriching, sensitive data, routing and reduction, detection, alert triage, observability, AI agent and LLM telemetry, pipeline and data health, and vulnerability context. Macro accuracy averages over the 39 task types.

General holdout: 26 public classification tasks (ade, arena_pref, bbc_news, cb, cr_reviews, dbpedia_l2, dbpedia_l3, dolly_category, fin_phrasebank, fin_sentiment, hermes_tools, hwu64, massive_scenario, paws, pubmedqa, quality, quality_full, reward_bench, sciq, social_iqa, strategyqa, student_questions, trec, truthfulqa, tweet_irony, xstory_cloze), up to 300 test questions each. It measures how much general ability the model keeps; per-task results are in evaluation.json.

Intended use and limitations

• Built for short, structured decisions over machine data in English. It is not a chat model and does not explain its answers.
• Where latency or memory matters more than accuracy, Cribl Decision 2 Mini is much smaller and faster, but less accurate (0.728 on the telemetry benchmark).
• Probabilities are reasonably calibrated on the benchmark above, but treat them as scores, not guarantees, particularly on data unlike IT and security telemetry.
• One question per forward pass; the options must fit in the 4,096-token prompt together with the state.
• Text inside the state is treated as data. Special tokens in user text are neutralised, but as with any model, do not rely on it as a security boundary against adversarial input.

License

Apache License 2.0 (see LICENSE and NOTICE). The backbone is derived from Qwen3.5-4B (Apache 2.0).