decider-35b-a3b: typed decisions with calibrated probabilities in one forward pass, 35B mixture of experts

A language model that does not generate text. It reads a state and one or more typed questions, each with an explicit option list, and returns a probability distribution over the options for every question from one forward pass. There is no decoding, no parsing and no output outside the options you defined. It is called from software, not chatted with. It is an open reproduction of the "System One" model class (TypeSafe AI's Jev).

Base model: Qwen/Qwen3.5-35B-A3B-Base: 34.7B parameters, of which 3B are active per token (256 routed experts, 8 active, plus a shared expert; 40 layers, 10 with full attention and 30 with gated delta-net linear attention). The supervised recipe of decider-2b (one epoch of cross-entropy on the slot readout over the public decision mixture) was applied to it with the routed experts frozen and the Muon optimizer on the block matrices. This repository holds v1, the bf16 weights (65 GB). An NVFP4 checkpoint of the same weights for vLLM and TensorRT-LLM is at Mapika/decider-35b-a3b-nvfp4; the smaller models are listed under The decider family. decider/ in this repository is the inference subset of the GitHub package.

Against decider-2b v10 on the same rows: accuracy is higher on 93 of the 95 regression tasks (in-task 0.855 against 0.805, held-out 0.810 against 0.755), +6.7 points on the 847 validation rows, +5.0 on OpenJev, +6.9 on Mind2Web, +5.9 on the TypeSafe workflow rows, JevBench hard tier 0.676 against 0.459, Bespoke's public suite 0.774 against 0.704 macro. Negative log-likelihood drops by 0.12 to 0.24 nats on every fixture. The model was not RL-trained: on live browser tasks its greedy play beats v10 (97.2% against 90.9%) and its sampled play is behind (86.4% against 93.2%). Details under Evaluation.

Contents: The decider family · Usage · How it works · Training · Evaluation · Speed · Limitations · Changelog · Reproduction

The decider family

All six repositories share one interface (decider.infer.Decider, POST /v1/systemone in TypeSafe's format) and one readout: the letter logits at an answer slot, softmaxed over the options. Pick by size and input.

Code, data registry, training scripts, the changelog and the per-version history: https://github.com/Mapika/decider.

Usage

from decider.infer import Decider          # decider/ is included in this repo
d = Decider("Mapika/decider-35b-a3b", use_graphs=False)
d.decide("My card was charged twice for the same purchase.",
         [{"question": "Which department should handle this?", "options": ["billing", "technical support", "sales"]},
          {"question": "Does this need a refund action?", "options": ["no", "yes"]}])
# [{'choice': 'billing', 'confidence': 0.99, 'probs': {...}}, {'choice': 'yes', 'confidence': 0.98, 'probs': {...}}]

The API is the same as decider-2b's: decide_batch scores many states with many questions in one call, abstain_below=t returns None under a confidence threshold, a question can have 2 to 255 options, and system_one / decider.serve accept TypeSafe's POST /v1/systemone request shape (the official typesafe-sdk works with TYPESAFE_BASE_URL pointing at the server). Every question and every Score level is scored in its own row. The state may be a string, object or array of up to 32k tokens. See the decider-2b card for the full description of the request shape, field types and the schema cache.

Requirements: one GPU with at least 80 GB of memory (the weights take 65 GB in bf16), torch>=2.14, transformers>=5.17 and flash-linear-attention. config.json sets experts_implementation: grouped_mm, which runs the 256 experts of a layer as one grouped matrix multiplication; the eager expert loop that transformers falls back to on older versions is about 13x slower. use_graphs=False is required: the CUDA-graph engine and the FP8 path of the helper package were built for the dense models and are untested with this architecture. Loading takes about 25 seconds from local disk.

Apple Silicon (reported by @nassersala in issue #6): on an M5 Max with 128 GB of unified memory, macOS 26.5, torch 2.14 and transformers 5.17, the model loads in bf16 in about 60 s through Decider(path, device="mps", dtype=torch.bfloat16, use_graphs=False) and reproduces this card's JevBench public-item counts exactly (48/48, 70/72, 75/111). flash-linear-attention does not install there; install the package with --no-deps after torch, transformers>=5.17, numpy<2 and huggingface_hub. From decider-ai 1.1.4 the library's MPS patch replaces two slow MPS operations in the MoE reference code (decider/mps_moe.py), which takes a decision from 2.8-4 s to 0.23-0.5 s for typical inputs.

Without the helper package, the same computation in plain transformers:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained(REPO); m = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16).cuda().eval()
prompt = ("Context:\nMy card was charged twice for the same purchase.\n\n"
          "Question: Which department should handle this?\nOptions:\n(A) billing\n(B) technical support\n(C) sales\nAnswer: (")
ids = tok(prompt, return_tensors="pt").to("cuda")
with torch.no_grad():
    logits = m(**ids).logits[0, -1]
letters = [tok.encode(L, add_special_tokens=False)[0] for L in "ABC"]
probs = torch.softmax(logits[letters].float() / 1.08, -1)      # 1.08 is the stored temperature

How it works

The prompt is Context: ... followed by, for each question, the question text, the lettered options (A) ... (B) ... and an answer slot Answer k: (. The hidden state at each slot is projected with the option-letter rows of the LM head and softmaxed over the valid letters, divided by the temperature in decider_config.json. Letters are never generated, so all slots are read from one pass. Large label sets were sub-sampled to at most 10 options per training example (gold always kept, order shuffled), so the model conditions on the supplied candidates rather than on a fixed head.

Training

One epoch of the public supervised mixture of the GitHub repository (scripts/train.sh full: about 95 public decision datasets, agent trajectories, Mind2Web element choice, game states, teacher-written custom questions, Jev's input shapes, two prompt layouts, isolated Score levels, 10% abstention rows). 1,543,567 items, 463M tokens, pre-tokenized once and read in the same order by every rank.

Muon was chosen over AdamW on a same-data comparison stopped at 11% of the epoch: at every logged step both optimizers had seen identical examples, and Muon's cross-entropy was lower in 75 of 94 logged windows, 0.537 against 0.605 at step 1,880 (mean over steps 1,500 to 1,880: 0.557 against 0.614). No AdamW model was trained to the end, so there is no AdamW row in the evaluation tables. The temperature 1.08 was fitted on the in-task half of the regression set. No reinforcement-learning stage was run on this model; the RL recipe of decider-2b v10 is documented in docs/RL.md of the GitHub repository.

Evaluation

Public regression set, rebuilt on this machine (95 tasks: 67 in-task, 28 held-out; large label sets sub-sampled to 10 options; one temperature per model fitted on in-task data). The decider-2b rows are the same set, same rows. ECE is the expected calibration error with 15 bins.

Half the epoch reaches 99% of the final in-task accuracy; held-out accuracy is flat from 50% to 100%. Accuracy is above v10 on 93 of the 95 tasks and 0.6 points below on two (counterfactual detection, offensive-tweet detection). The largest gains are on knowledge and reasoning tasks: MedQA +31 points, MedMCQA +24, TruthfulQA +22, Winogrande +20, MMLU +19, StrategyQA +19.

Per-task accuracy / ECE on the 28 held-out datasets, decider-2b v10 against this model

Per-task accuracy / ECE on the held-out datasets, v10 against this model:

On the same rows as decider-2b. Every row below is scored by both models on identical inputs and seeds. Intervals are 95% paired bootstrap intervals.

The browser rows show what the RL stage of v10 does and this model lacks: v10's sampled play matches its greedy play because RL sharpened the served distribution on those tasks; this model's argmax is right more often, but its distribution still puts mass on wrong elements (its sampled play is +3.4 points against v8, which had no RL either). Among the games, the largest greedy gains are on the slippery grid (+17 points) and tic-tac-toe (+14); minesweeper stays near zero for every model.

JevBench public items (231 items of Benchmark Heaven; argmax over the exact label set with the request the harness's TypeSafe adapter builds). Jev 1.13.0 is at 1.000 / 0.986 / 0.730, djev at 1.000 / 0.986 / 0.676, SemIf 4B at 1.000 / 0.986 / 0.613 on the same items, from their published per-item outcomes. This model's hard-tier misses are on temporal-numeric items (0.33), long policies (0.63) and judge-hard items (0.65); adversarial, trap and hard routing items are all correct. Top-label ECE is 0.001 / 0.059 / 0.151 by tier: the model is overconfident on the hard tier.

Bespoke's public suite (13 human-labelled subsets, 3,880 records in Jev's wire format, answered through system_one as shipped). Nimble-9B and Jev 1.13.0 numbers are copied from Bespoke's report.

On the six subsets whose training split is not in the mixture the macro accuracy is 0.744. The model is behind Jev where a claim has to be checked against evidence that nearly matches it (PAWS, SummEval consistency) and on SQuAD2 answerability.

Behaviour probes (teacher-labelled, same probes as the 2B releases): generic-versus-specific bucket choice 1.00 / 1.00, catch-all when nothing fits 0.95; command-risk classification 0.933 with no destructive command called safe; browser-agent element and action choice 0.938 / 0.938. Scoring a Score level alone against scoring it with its neighbours changes accuracy by at most 2 points on five rating datasets, and the per-level fits sum to between 0.92 and 1.07.

Speed

One NVIDIA B300, bf16, eager PyTorch (use_graphs=False), grouped-GEMM experts. A 92-token support ticket with three typed questions, and a 5-token chat message with one question:

decider-2b serves the same tickets at 4 ms with CUDA graphs and about 1,400 to 2,100 decisions/s; this model is for workloads where the accuracy gain is worth 3 to 4 times the cost per decision, and for the NVFP4 build on Blackwell (see the -nvfp4 repository).

Limitations

• No reinforcement-learning stage: stated beliefs about action outcomes were not trained against exact laws, and the served distribution on live browser tasks is less sharp than decider-2b v10's (sampled play 86% against 93%).
• 65 GB of bf16 weights; one 80 GB GPU is the minimum on CUDA (a Mac with 128 GB of unified memory also runs it, see Requirements), and the CUDA-graph and FP8 paths of the helper are untested here.
• Overconfident on the hardest external items (JevBench hard-tier ECE 0.15) and on some held-out classification sets (TREC 0.16, financial sentiment 0.14) although the aggregate ECE is 0.03 / 0.07.
• English only. Calibration is measured on public datasets and teacher-labelled probes, not on your traffic. Check it on your own labels before using confidence for routing.
• The routed experts are the base model's: the fine-tuning changed 2.45B of the 34.7B parameters.
• Everything else in the decider-2b card's limitations (packed questions see each other, long JSON arrays by position, full label sets against sampled options, abstention wording) applies; those shapes were not re-measured at this size.

Changelog

The GitHub repository's docs/CHANGELOG.md lists every decider release.

Reproduction

Code, data registry, training and evaluation scripts and the per-version history: https://github.com/Mapika/decider (docs/HISTORY.md, section "decider-35b-a3b"). The training code for the frozen-expert Muon run is in the repository's history document; the merged checkpoint is this repository. Staged with scripts/stage_release.py and uploaded with scripts/upload_hf.py.