decider-31b: typed decisions with calibrated probabilities on Gemma-4-31B, from text and images, one pass

decider-31b answers typed decisions with a probability for every option. It reads each question in one forward pass and generates no text. From v2 the state may include images.

• Input: a state and one or more typed questions (Choice, Noul yes/no, Score), each with an explicit option list, in the System One wire format (POST /v1/systemone). Images are sent as Jev-style assets and referenced from the state.
• Output: a probability distribution over the options of every question. Nothing needs parsing.
• Weights: google/gemma-4-31B-it with a fine-tune merged into the attention and MLP projections, quantized to NVFP4 (MLP weights and activations in NVFP4, attention projections and the vision tower in bf16, FP8 KV cache). About 31 GB.
• Code: decider.serve_vllm in decider-ai 1.10.0. It reads this repository's decider_config.json, which holds the configuration measured below and switches image input on.
• No thinking. This model never generates a reasoning trace. Every request is answered in one or two forward passes.
• Versions: this repository holds v2 (2026-10-20). v1, the text-only model of 2026-10-07, stays available under the Hub tag v1.

Changes from v1

v2 is v1 trained further on image decisions, with a part of v1's text data replayed. Measured on the same suites with the same server:

• The index score is 0.33 lower. The loss is in Retrieval & Classification (65.53 to 62.47); Knowledge & Reasoning and Language Understanding are higher (50.35 to 50.87, 66.60 to 67.32).
• Calibration on the index is better: ECE_bw 0.018 against 0.035.
• The image gain is on the kinds of image questions it was trained on (for example Hateful Memes 0.72 to 0.83, RAVEN 0.25 to 0.47). On the two image sets it never saw, it is level.
• Use v1 (revision="v1") if you need text only and the last third of a point on the index.

Results

Decision Index 0.3, public suite, all 140,178 requests (kit --edition 0.3), run by us on 2026-10-19 with our development server (the same readout as decider.serve_vllm; on v1 the two gave 64.23 and 64.27). The board's full score also includes private tests that only the maintainers run; this is the public part only.

Image questions: the eval splits of 11 sets from the_cauldron, 300 items each, read the same way as text questions:

Through decider.serve_vllm from decider-ai 1.10.0, with the image sent as a Jev-style asset, v2 gives 0.893 on Visual7W and 0.840 on VSR on the same items, and 61.69 on our 4,500-request index sample (development server 61.78).

Notes on the numbers:

• The run was not verified by the Decision Index maintainers.
• ECE_bw is the benchmark-weighted expected calibration error over the 214,304 scored fields whose rows are in the suite's selected-rows file.
• The temperatures were fitted on our own rows only. No Decision Index row and no image evaluation item was used to fit or choose them.
• Training used the public train splits of several benchmarks that the index also tests (see Training data). No Decision Index test row was used: every training row was compared with all suite rows and dropped on a match.

Latency

Not measured again for v2. The text path is the same model size, quantization and readout as v1, which measured median 39 ms, mean 116 ms, p80 203 ms and p95 531 ms on one B300 (one request at a time, 400 Decision Index requests). The share of questions read a second time can differ from v1's. Requests with images were not timed.

Usage

vLLM 0.29.0 pins its own torch, so install in a separate environment:

python -m venv decider-vllm && . decider-vllm/bin/activate
pip install vllm==0.29.0 fastapi "uvicorn[standard]" jinja2 huggingface_hub pillow
pip install --no-deps "decider-ai>=1.10.0"
DECIDER_MODEL=Mapika/decider-31b uvicorn decider.serve_vllm:app --host 127.0.0.1 --port 8000

It needs one GPU with NVFP4 support (Blackwell: B200/B300, RTX PRO 6000, RTX 50 series with enough memory).

A text request:

import httpx
r = httpx.post("http://127.0.0.1:8000/v1/systemone", timeout=60, json={
    "state": {"ticket": "Order #1182 arrived broken. Customer asks for a refund; order is 40 days old; policy: 30 days."},
    "questions": {"refund": {"type": "noul", "instructions": "Is the refund allowed under the policy?",
                             "criteria": {"true": "allowed", "false": "not allowed"}}}})
print(r.json()["answers"]["refund"])

A request with an image. The asset's key is the reference; every place the reference appears in the state is replaced by the image:

import base64, httpx
img = base64.b64encode(open("shelf.jpg", "rb").read()).decode()
r = httpx.post("http://127.0.0.1:8000/v1/systemone", timeout=60, json={
    "state": {"photo": "shelf-photo", "note": "Restock when fewer than 3 boxes are left."},
    "assets": {"shelf-photo": {"type": "image", "format": "jpg", "data_base64": img}},
    "questions": {"restock": {"type": "noul", "instructions": "Does the shelf in `photo` need restocking?",
                              "criteria": {"true": "restock", "false": "enough boxes"}}}})

Images are PNG or JPEG. A reference must appear in the state; references in instructions or criteria are not supported yet, and an asset that the state never names is rejected (HTTP 422). DECIDER_IMAGES=0 turns image input off and starts vLLM with the language model only.

How a request is answered

1. First reading. Every question is rendered in the model's chat template with thinking off, and the log-probabilities of the option letters are read at the answer slot. A softmax at T1(n) = 2.064 − 0.053 ln n (n = number of options) gives the probabilities. Each image in the state is one placeholder, which vLLM expands to the image's soft tokens (at most 280).
2. Second reading, only when unsure. A question whose top probability is below 0.7 is read again with its options in reversed order. The two log-softmaxes are averaged and a softmax at T2(n) = 1.904 − 0.022 ln n gives the final probabilities.
3. Questions above 0.7 keep the first reading.

The thresholds and temperatures are in decider_config.json (temperature_by_options, second_reading, images). DECIDER_SECOND_READING_BELOW=0 turns the second reading off. The config also selects vLLM's Triton attention backend (vllm_attention): with this checkpoint's FP8 KV cache, the FlashInfer backend returned NaN in batched readouts in our tests.

Training data

v2 starts from v1 and adds a LoRA stage (rank 32, learning rate 2e-5, 421 steps, 12.3M tokens, 2 hours on one B300) on:

• Image decisions: 18,129 rows from the train splits of seven the_cauldron subsets: AI2D 1,939, A-OKVQA 3,000, Hateful Memes 3,000, IconQA 3,000, RAVEN 3,000, ScienceQA 3,000, TQA 1,190. Yes/no questions are trained as Noul questions, the others as Choice questions. Visual7W and VSR were not used, so they measure image questions the model never saw.
• Text replay: 5,000 rows drawn from v1's training data.

v1's training data, unchanged:

• rows from public train splits of tasks where our previous entry was weak, rendered in the Decision Index request format, and decontaminated against every Decision Index 0.3 suite row: CLINC150 (train), iSarcasmEval (task A, English), VAST (train), ACOS (laptop and restaurant, train), POP909-CL (songs not used by the index), GSM8K (train), NLI4CT (train);
• generated smart-home requests from our own generator;
• from our earlier decider models: generated state-tracking decisions and decision families written by our own code, document questions written by Qwen3.6-27B, and labelled rows from public datasets (MMLU, ARC, CommonsenseQA, BoolQ, MNLI, SNLI, BANKING77, RACE, OpenBookQA, LogiQA 2.0, MedQA, WinoGrande).

Not used: Decision Index test rows, JevBench or ImageJevBench items.

Limitations

• One pass only. Tasks that need several steps of reasoning (GPQA, ChessBench, HLE on the index) are not solved by reading harder.
• Images: the gain is on the kinds of image questions trained (diagrams, science figures, icons, memes, puzzles). On image question types it never saw, it reads about as well as v1 and the stock model. Box answers (Jev's box type) are not supported. We have not measured it on ImageJevBench.
• Retrieval & Classification is 3 points lower than v1 on the index.
• Calibration: check it on your own data before relying on the probabilities.
• Hardware: NVFP4 needs a Blackwell GPU. We measured only on B300.
• vLLM version: the server patches a vLLM 0.29 internal and refuses other versions.
• Independent questions only: /decide and the schema cache of decider.serve are not served.

License

The weights are released under Apache 2.0, as the base model. Training sources and their terms:

• Image questions, from the train splits of the_cauldron subsets (the_cauldron licenses only its own prompts, CC BY 4.0; each source keeps its own terms):

  • AI2D (Kembhavi et al., 2016): CC BY-SA 4.0.
  • A-OKVQA (Schwenk et al., 2022): Apache 2.0 (repository); images from COCO.
  • IconQA (Lu et al., 2021): CC BY-NC-SA 4.0, non-commercial.
  • ScienceQA (Lu et al., 2022): CC BY-NC-SA 4.0, non-commercial research use.
  • TQA (Kembhavi et al., 2017): CC BY-NC 3.0, non-commercial.
  • RAVEN (Zhang et al., 2019): no data licence stated (the code is GPL-3.0).
  • Hateful Memes (Kiela et al., 2020): Facebook's Hateful Memes dataset licence. It permits use of the data to research and develop models designed to detect manipulated media, and states that models derived from it may be used for academic and commercial purposes. This is a research project; check the licence for your own use.

• CLINC150: CC BY 3.0 (Larson et al., 2019). Credit: "An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction", clinc/oos-eval.

• iSarcasmEval: MIT (Abu Farha et al., 2022). The texts are tweets.

• POP909-CL: MIT; built on POP909 (Wang et al., 2020), MIT.

• VAST (Allaway and McKeown, 2020) and ACOS (Cai et al., 2021): no licence stated by the authors.

• GSM8K (Cobbe et al., 2021): MIT.

• NLI4CT (SemEval-2024 Task 2, Jullien et al.): no licence file in the task repository; released for the shared task.

• RACE: non-commercial research use. It is a small part of the replayed data (250 rows).

• ARC, SNLI, LogiQA 2.0 (copy dependent): CC BY-SA 4.0; BoolQ: CC BY-SA 3.0; BANKING77, MedQA: CC BY 4.0; MMLU, CommonsenseQA: MIT; MNLI: mixed per genre.