Torchcast Decision 31B (v1.0)

A text-based decision model developed by Torchcast AI, based on Gemma-4-31B-it. It returns yes/no probabilities (noul), choice distributions (choice) and ordinal score distributions (score) through a TypeSafe-compatible API. Each decision reads the option letters at the answer position in one forward pass (choice lists with more than 26 options are decided in a few passes over groups of options); nothing is generated.

Use the bundled torchcast_decision serving package with the bundled decision_config.json. Generic Hub text-generation examples and causal-attention runtimes (vLLM, llama.cpp, MLX) do not reproduce the evaluated interface: the global attention layers attend to the whole prompt in both directions (see below).

Serving instructions

Model

• Weights: a full fine-tune of google/gemma-4-31B-it (all layers except the input embeddings) on a mix of public datasets (official training splits) and synthetic decision data covering choice, yes/no and score questions. No benchmark test items were used for training.
• Attention: hybrid. The sliding-window layers are causal as in the base model; the global layers use no causal mask within a prompt (attention_mode: hybrid_bidir_global), as in training.
• Prompt format: torchcast_gemma4_v1 (bundled torchcast_decision/prompt.py), Gemma 4's chat format with thinking off.
• Serving settings (decision_config.json): temperature 0.8 (choice), 0.6 (yes/no), 0.6 (score); yes/no commit floor 0.801.

Results

Board results pending: the model has not yet been measured on the JevBench leaderboard.

Artifact

• Pinned revision: tag v1.0.0 of this repository.
• Weight SHA-256: see SHA256SUMS.
• Served model name: torchcast-decision-31b-v1.0.

Serving settings

Serve the model through the bundled torchcast_decision package (Torchcast AI, Apache License 2.0); decision_config.json holds the serving settings. Within a prompt, the global attention layers use no causal mask (as in training), so loading the weights with plain transformers, vLLM, llama.cpp or MLX gives different probabilities. The bf16 weights need one GPU with at least 80 GB: tested on an H100 80 GB (FlashAttention-3 kernel for the sliding-window layers, pinned to a fixed commit and prefetched for offline starts) and an RTX PRO 6000 96 GB (PyTorch SDPA kernel). Prompts longer than 16,384 tokens keep the head and the tail of the state, as in training. At start-up the server captures CUDA graphs for prompts up to 4,096 tokens (about one minute); graph and eager outputs are bit-identical. Repeated runs on the same GPU type give identical probabilities. One request at a time over the public JevBench items, the median latency is 48 ms on the H100 and 74 ms on the RTX PRO 6000; the longest prompt takes 4.1 s and 6.2 s. See NOTICE and the serving instructions on GitHub.

Intended use and limitations

English text-based decision research, such as classification, routing and bounded rubric evaluation. The server returns decisions and does not execute actions. Probabilities can be unreliable on new domains.

Licence and attribution

Model weights: Apache License 2.0 (see LICENSE). Base model: Gemma-4-31B-it by Google, Apache License 2.0 (NOTICE, Gemma 4 licence). Serving package: Apache License 2.0 (torchcast_decision/LICENSE, torchcast_decision/NOTICE). This project is independent of TypeSafe AI and the JevBench maintainers.