Cerno 4B v1.2

Jev-style System One decision model · 4B · one forward pass · calibrated probabilities · block attention · read twice

Cerno 27B · Cerno 9B v1.2 · Cerno 4B v1.2 · Inference code · Decision Index runs

v1.2 (2026-10-30): a new 4B, trained from the post-trained Qwen3.5-4B with block-bidirectional attention and served with read twice. +7.45 on the public index over v1.1 (57.60). It runs on the transformers engine of the inference code, in bf16. Cerno 4B v1.1 (FP8, vLLM) stays at apertixai/cerno-4b.

What it does

• Input: a state (any text or JSON) and a set of typed questions about it.
• Question types: multiple choice with explicit criteria, yes/no, scored scales. Up to 255 options per question.
• Output: a probability for every option of every question, in one forward pass with no text generation: the state and all questions are packed into one prompt.
• Block attention: in the full-attention layers every token attends both ways within its block (the state, or one question with its options) and causally to the blocks before it. vLLM cannot apply that mask, so the model runs on transformers.
• Read twice: a request of up to 3,000 tokens is read twice: the state, the questions with their options, "Read the same input again before answering:", then the state again (--option read_twice_tokens=0 turns it off).
• Calibrated: one shipped temperature (1.236), fitted on held-out sets, rescales the probabilities (ECE 6.3 pts on the board's measure, under Results); it changes no answer.
• Speed: 61 ms median and 208 ms mean per request on one H200, one request at a time.

The Cerno family

• Public index: the complete public suite (37 benchmarks, 140,178 requests) run with the maintainers' kit at commit 9eb2dbe through the public inference code and these weights.
• The board ranks by its Full score: private tests of the same skills (60%) and private tasks from new domains (40%), run by the maintainers. The public index does not count toward it. v1.2's Full score is pending.

Results

How to read the rows

• Strict-OOD drops HLE, BANKING77, CLINC150 and HellaSwag, whose public train splits are common training data.
• Train-split-OOD drops every benchmark family whose public train split we used (listed under Training data).
• ECE is the board's definition: expected calibration error over a 1-in-6 sample of 34 benchmarks, equal weight per benchmark, at the shipped temperature.
• Run-to-run noise on the public index is about 0.1-0.2 points. The public index is not the board's ranking.

Quick start

git clone https://github.com/apertixsk/cerno-inference && cd cerno-inference
pip install -r requirements-v1.2.txt && export PYTHONPATH=$PWD:$PWD/third_party/decider
bash run_sample.sh apertixai/cerno-4b-v1.2       # reproduces 60 requests of our submitted run

The complete public suite, exactly as submitted:

python -m decision_index pipeline --edition 0.3 --engine evals.decision_index.engine:CernoEngine \
    --option model=apertixai/cerno-4b-v1.2 --option revision=<commit> --out runs/cerno-4b-v1.2

• Weights: bf16 safetensors, 8.4 GB. One GPU with 24 GB or more.
• Settings in decider_config.json: attention: block, read_twice_tokens: 3000, temperature: 1.236; the engine applies all three. The temperature was fitted on held-out, non-benchmark eval sets and is argmax-invariant.
• Software: transformers 5.17.0 (its block mask) with the flash-linear-attention kernels, as pinned in requirements-v1.2.txt.

Examples

Requests from our own evaluation set (not benchmark data), answered by this model through the public engine.

Agent guardrail: should this tool call run without asking?

state = "rm -rf /tmp/build-cache-7f3a2e   # user asked to clear the build cache before a rebuild"
questions = {
    "destructive": {"type": "noul", "instructions": "Would executing this tool call delete, overwrite or irreversibly change data?"},
    "matches_request": {"type": "noul", "instructions": "Is this tool call a reasonable way to do what the user asked?"},
    "external_effect": {"type": "noul", "instructions": "Does this tool call affect people or systems outside the local machine (sends messages, moves money, publishes)?"},
    "risk": {"type": "score", "instructions": "How risky is it to run this tool call without asking the user first?", "criteria": ["harmless, read-only", "minor, easily reversible", "significant or hard to reverse", "severe or irreversible"]}
}

Support triage: a one-line customer message

state = "hey do u have showers at the downtown location"
questions = {
    "kind": {"type": "choice", "instructions": "What kind of message is this?", "criteria": {"incident": "an outage or something broken in production", "billing": "invoices, charges, refunds", "account": "cancellations, plan changes, access", "feedback": "praise or suggestions", "spam": "unsolicited or scam", "other": null}},
    "urgency": {"type": "score", "instructions": "How urgent is this message?", "criteria": ["no action needed", "can wait a few days", "should be handled today", "needs attention right now"]},
    "tone": {"type": "score", "instructions": "What is the sender's tone?", "criteria": ["hostile", "negative", "neutral", "positive"]},
    "needs_human": {"type": "noul", "instructions": "Does this message need a human to respond rather than an automated reply?"}
}

Programmatic use, with the repository on PYTHONPATH:

from evals.decision_index.engine import CernoEngine

engine = CernoEngine(model="apertixai/cerno-4b-v1.2")   # block attention, read twice and the temperature from decider_config.json
response, _ = engine(state, questions)                      # one forward pass over the state and every question

Question types: choice (criteria = option descriptions, null for none), noul (yes/no), score (ordered level descriptions).

Serving and latency

• One H200, one process, one request at a time, the kit's 400-request stratified sample: median / mean / p80 61 / 208 / 266 ms, p95 920 ms.
• Over the whole public suite (the submitted run, eight processes per node): median / mean / p80 56 / 97 / 86 ms.
• The public code gives the same answers as our development engine on all 3,165 answer fields of that sample.
• Not measured on the board's reference hardware (one RTX PRO 6000).

Training data

Full fine-tune of Qwen/Qwen3.5-4B. No Decision Index test row and no HLE row was used; the training records were screened against the requests of the 0.3 suite before training.

• A public instruction / classification mixture.
• The public TRAIN/DEV splits (never test rows) of these Decision Index benchmark families: BANKING77, CLINC150+OOS, ANLI, VAST, WinoGrande, GSM8K, HoVer, RAGTruth, ContractNLI, Humicroedit, ACOS, Amazon ESCI, NLI4CT, When2Call, New Yorker caption matching, PhishNChips, Home appliances, Habermas Machine comparisons.
• Synthetic decision items across many domains, written and labelled with an external teacher model, DeepSeek-V4.1-Flash-1017 (MIT licence); its answer distributions are used as soft labels.
• Procedurally generated items modelled on the formats of some public benchmarks, with no benchmark text.

The strict-OOD and train-split-OOD rows above drop the affected benchmarks.

Licence and credits

• Weights and code: Apache-2.0. Base model: Qwen/Qwen3.5-4B, Apache-2.0.
• The prompt layout and label-token readout follow the open decider recipe (Apache-2.0); its inference subset is vendored in the code repository.
• Block-bidirectional attention follows the open OneJev / Mammoth readout (Apache-2.0 / MIT).
• Read twice uses the layout of KnowLine Gen5.5 and FlexDM (MIT), after RE2 (Xu et al. 2023, arXiv 2309.06275).
• Training labels include outputs of DeepSeek-V4.1-Flash-1017 (MIT).
• Benchmark data stays under its own terms. Jev and System One are TypeSafe AI's names; this project is not affiliated with TypeSafe AI or with the Decision Index maintainers.