decider-2b: typed decisions with calibrated probabilities in one forward pass, 2B dense, text and images

A language model that does not generate text. It reads a state and one or more typed questions, each with an explicit option list, and returns a probability distribution over the options for every question from one forward pass. It is called from software, not chatted with. From v12 the state may include images.

This repository holds v12 (2026-10-20). v11 stays available under the Hub tag v11, with its own card; v10 under v10.

• Weights: the text model at the repository root (Qwen3.5-2B-Base architecture, bf16, 3.8 GB), loaded as before by decider.infer.Decider, decider.serve and the GGUF builds. vision/ holds the image part: the vision tower of Qwen3.5-2B-Base (unchanged, 0.6 GB) and the config of the image-text model. The image path reuses the text model's weights.
• Code: decider-ai 1.10.0 for images; text works with 1.4.0 and later.

Changes from v11

• Decision Index: +10.1 on the sample, from the decider-31b training mixture (train splits of benchmarks the index also tests; see Training).
• Images: new. The gain over the plain base vision tower is mostly on the kinds of image questions trained.
• JevBench public items: two hard items more, two standard items more.
• Yes/no answers: the config now uses temperature 0.05 for Noul answers, so they come out near 0 or 1. JevBench counts a yes/no probability between 0.2 and 0.8 as wrong; at 0.05 almost none fall there, and our 399-item held-out yes/no set scores 29.6 instead of −40.1 at the fitted temperature. The cost: the yes/no probabilities are no longer calibrated (ECE 0.341 instead of 0.095 on that set). For calibrated yes/no probabilities use temperature_by_type={"choice": 2.465, "noul": 3.185, "score": 2.833} (the fitted values).

The decider family

All share one interface (POST /v1/systemone in TypeSafe's format) and one readout. Code and changelog: https://github.com/Mapika/decider.

Usage

Text, as before:

from decider.infer import Decider
d = Decider("Mapika/decider-2b")
d.decide("My card was charged twice for the same purchase.",
         [{"question": "Which department should handle this?", "options": ["billing", "technical support", "sales"]},
          {"question": "Does this need a refund action?", "options": ["no", "yes"]}])

Images, through the server (decider-ai 1.10.0; the config's "images": true turns image input on):

pip install "decider-ai>=1.10.0" pillow
DECIDER_MODEL=Mapika/decider-2b uvicorn decider.serve:app --host 127.0.0.1 --port 8000

import base64, httpx
img = base64.b64encode(open("chart.png", "rb").read()).decode()
r = httpx.post("http://127.0.0.1:8000/v1/systemone", timeout=60, json={
    "state": {"chart": "q3-chart", "note": "Escalate if any region fell by more than 10%."},
    "assets": {"q3-chart": {"type": "image", "format": "png", "data_base64": img}},
    "questions": {"escalate": {"type": "noul", "instructions": "Does the `chart` call for escalation?",
                               "criteria": {"true": "escalate", "false": "no escalation"}}}})

The asset's key is the reference; each place the state names it becomes "the image" in the prompt, and the image goes in front of the prompt, as in training. PNG and JPEG; up to 4 images (DECIDER_MAX_IMAGES); references in question text and box answers are not supported. Image rows are read one at a time, without batching or CUDA graphs. Requirements as before (torch, transformers>=5, flash-linear-attention), plus pillow for images.

Training

v12 starts from v11. Its text weights were placed into the Qwen3.5-2B-Base image-text model (vision tower from the base); the language-model weights are identical to the text model's. Then a LoRA stage (rank 64, learning rate 1e-4, 2 epochs, 56 minutes on one B300) on the language model only:

• Text, 39.5k rows: the decider-31b training mixture (train splits of CLINC150, iSarcasmEval, VAST, ACOS, POP909-CL, GSM8K and NLI4CT, decontaminated against every Decision Index 0.3 suite row; generated smart-home requests; generated state-tracking decisions and decision families; document questions written by Qwen3.6-27B; labelled public rows), plus 6.7k rows replayed from the decider-4b v2.1 training data.
• Images, 18,129 rows: train splits of seven the_cauldron subsets: AI2D, A-OKVQA, Hateful Memes, IconQA, RAVEN, ScienceQA, TQA. Visual7W and VSR were not used.
• Not used: Decision Index test rows, JevBench or ImageJevBench items.

Temperatures were fitted on our own validation rows: Choice 2.465, Score 2.833, Noul 3.185 (the config uses 0.05 for Noul, see above). They are about twice v11's because v12's raw answers are sharper.

We also trained the same recipe for one epoch. Our own validation rows preferred it slightly (Choice NLL 0.722 against 0.764); the Decision Index sample (37.26 against 38.17) and the JevBench public items (hard 0.541 against 0.595) preferred two epochs, and we released the two-epoch run. So the choice between the two runs looked at benchmark readouts; nothing else did.

Evaluation

All numbers measured by us; none were verified by the benchmark maintainers.

• Decision Index 0.3, 4,500-request sample of the public suite: 38.17 (v11 28.05); through decider.serve 1.10.0, 38.18. Benchmark-weighted ECE on the sample: 0.164 (v11 0.161); with about 88 requests per benchmark this number is inflated for every model. The full public suite was not run.
• Image questions, eval splits of 11 the_cauldron sets, 300 items each: mean 0.847; trained kinds 0.864, never-trained (Visual7W, VSR) 0.790. v11 with the base vision tower and no image training: 0.778 / 0.788. Stock Qwen3.5-2B with our readout: 0.774 mean. Through decider.serve 1.10.0 with the image sent as an asset: VSR 0.753, Visual7W 0.853.
• JevBench public items (read once, never used for training, fitting or selection): easy 1.000, standard 0.917, hard 0.595.

Limitations

• One pass only: knowledge stays at the base model's level; multi-step tasks are not solved by reading harder.
• Yes/no calibration is traded for JevBench's rule in the default config (see Changes).
• Images: new image question types read about as well as the stock model. Not measured on ImageJevBench.
• Latency was not measured again; the text path is unchanged from v11.

License

The weights are released under Apache 2.0, as the base model. Training sources and their terms:

• Image questions, from the train splits of the_cauldron subsets (the_cauldron licenses only its own prompts, CC BY 4.0; each source keeps its own terms):

  • AI2D (Kembhavi et al., 2016): CC BY-SA 4.0.
  • A-OKVQA (Schwenk et al., 2022): Apache 2.0 (repository); images from COCO.
  • IconQA (Lu et al., 2021): CC BY-NC-SA 4.0, non-commercial.
  • ScienceQA (Lu et al., 2022): CC BY-NC-SA 4.0, non-commercial research use.
  • TQA (Kembhavi et al., 2017): CC BY-NC 3.0, non-commercial.
  • RAVEN (Zhang et al., 2019): no data licence stated (the code is GPL-3.0).
  • Hateful Memes (Kiela et al., 2020): Facebook's Hateful Memes dataset licence. It permits use of the data to research and develop models designed to detect manipulated media, and states that models derived from it may be used for academic and commercial purposes. This is a research project; check the licence for your own use.

• CLINC150: CC BY 3.0 (Larson et al., 2019). Credit: "An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction", clinc/oos-eval.

• iSarcasmEval: MIT (Abu Farha et al., 2022). The texts are tweets.

• POP909-CL: MIT; built on POP909 (Wang et al., 2020), MIT.

• VAST (Allaway and McKeown, 2020) and ACOS (Cai et al., 2021): no licence stated by the authors.

• GSM8K (Cobbe et al., 2021): MIT.

• NLI4CT (SemEval-2024 Task 2, Jullien et al.): no licence file in the task repository; released for the shared task.

• RACE: non-commercial research use. It is a small part of the replayed data (250 rows).

• ARC, SNLI, LogiQA 2.0 (copy dependent): CC BY-SA 4.0; BoolQ: CC BY-SA 3.0; BANKING77, MedQA: CC BY 4.0; MMLU, CommonsenseQA: MIT; MNLI: mixed per genre.

• The sources of v11 and earlier, listed on the v11 card (Hub tag v11).