Sieve-31B-v3

Sieve-31B-v3 is a decision model from the Sieve family. You give it a piece of text or JSON (the state) and typed questions, and it returns a calibrated probability for every option of every question. It reads the state once, scores the options directly, and never generates text.

It is a full fine-tune of google/gemma-4-31B-it (revision 842da37). It answers through its own output layer: the logits of the option labels (A, B, …) at an Answer: ( slot. Its 10 global attention layers are bidirectional, and all the questions of a request share one reading of the state. The code is at github.com/sthanika-ai/Sieve.

Ranked #1 (tied) on the Decision Index

Sieve-31B-v3 is ranked #1 (tied) on the Decision Index leaderboard, with a Full score of 71.10 (edition 0.4.1, board data of 2026-10-28, 243 models). The board ranks models by the Full score, which comes from the maintainers' private tests, and gives one rank to the models it counts as tied.

The Decision Index leaderboard on 2026-10-28.

Summary

How it works

• Prompt. Gemma's own chat template with thinking off: a system turn, then one user turn with the list of questions and the state, then one block per question with its labelled options, ending in an Answer: ( slot.
• Score. At each slot, the model's output-layer logits of the question's option labels (A, B, …, then AA, AB, …, up to 255 options; final logit softcap included). A softmax at the question category's temperature gives the probabilities.
• Two option orders. Every choice question whose options may be reordered is read twice, with its options as given and rotated by half; the two log-probability vectors are averaged. This cancels the preference for a label position. Questions marked "permute": false keep their order and are read once.
• Bidirectional attention. See the next section.
• Weights. A full Gemma-4 checkpoint in bf16 (13 safetensors shards), plus head.pt with the readout, the attention rule, the serving policy and the temperatures. The weights are a weighted average of the backbone (0.3) and the fine-tuned model (0.7).
• Never. It never truncates, drops or reorders options, tunes prompts per benchmark, or sees gold labels.

Bidirectional attention

Gemma-4-31B-it interleaves 10 global (full-attention) layers with 50 sliding-window layers (1,024-token window). A text-generation model has to be causal: each token only sees the tokens before it. A decision model generates nothing, so the global layers can read in both directions.

• One question. The whole prompt is one block: in the global layers every token attends to every other token, before and after it, so the state is read in the light of the question and its options.
• Several questions. The shared block (system turn, the list of questions, the state) attends only to itself. Each question's block attends to the shared block and to itself. Questions never see each other's options.
• Sliding-window layers. Causal in every case, as in the base model.
• Trained and served the same way. The fine-tune was trained with this attention, and sieve serves it the same way. head.pt records the rule, and sieve sets the attention when it loads the model.
• Shared state. In a request with several questions and a shared block of 2,048 tokens or more, sieve computes the shared block once and reuses its keys and values for every question (and both option orders). The result is the same as computing it with each question.

Use

pip install -r https://raw.githubusercontent.com/sthanika-ai/Sieve/main/requirements.txt   # the tested versions (torch 2.8.0)
pip install "sieve-decisions[serve] @ git+https://github.com/sthanika-ai/Sieve"

from sieve import load_sieve, decide

m = load_sieve("sthanika-ai/Sieve-31B-v3")
decide(m, {"subject": "Duplicate charge on invoice 4411", "body": "Billed twice. Refund today or we cancel."},
       {"department": {"type": "choice", "instructions": "Which team handles this?",
                       "criteria": {"billing": "invoices, payments, refunds", "technical": "bugs and outages"}},
        "churn_risk": {"type": "noul", "instructions": "Does the customer threaten to cancel?"}})

• Loading. Load it with sieve 0.4.0 or later, which reads head.pt and applies the prompt, the attention rule, the two option orders and the temperatures. Loading the weights with transformers alone gives the language model without any of these.
• GPU memory. The bf16 weights take 58.3 GiB. The serving benchmark observed a peak of 81.4 GiB allocated over 352 public Decision Index requests, including the largest by the kit's size estimate (17 questions; with this model's tokenizer, its shared block alone is 56,693 tokens and the prompt of its longest question 69,019). One 96 GB GPU runs every public request; shorter requests fit on an 80 GB GPU.
• Token limits. None declared. Requests are never truncated; one that does not fit in GPU memory fails with an error instead of a wrong answer. The model was trained on prompts of up to 16,384 tokens, and every public Decision Index request, the longest about 54,000 tokens, was answered.
• Server. sieve-serve --model sthanika-ai/Sieve-31B-v3.
• Decision Index. python -m decision_index run --engine sieve.decision_index_engine:SieveEngine --option model=sthanika-ai/Sieve-31B-v3 --out runs/Sieve-31B-v3.
• Temperature per category. head.pt stores a global T = 1.27 and 22 temperatures (0.955 to 1.380), fitted by NLL on the calibration split (17,753 questions) under the two-order reading. A question's category is its type, its number of options (2, 3-4, 5-9, 10-25, 26+) and the kind of state (empty, text or JSON). A question takes the temperature of its most specific category in the table (choice:3-4:text, then choice:3-4, then choice), else the global T. Temperatures never change an answer, and load_sieve(..., temperature=1.0) gives raw probabilities.

Results

Decision Index

Sieve-31B-v3 is ranked #1 (tied) on the Decision Index leaderboard (edition 0.4.1, board data of 2026-10-28), with a Full score of 71.10 and a public score of 71.68. On our own run of the public 0.3 suite with the official kit at f5a6e4f (0.3) and its scorer, it scores 71.68 (raw index 78.39, breadth 71.18).

• Coverage. All 140,178 scoreable requests over 43 benchmarks were answered, with no truncation, no unsupported request and no error. The index averages 37 of the benchmarks.
• How the run was made. One complete run of the 0.3 suite on eight NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs, one shard per GPU.
• Results. sthanika-ai/Sieve-31B-v3-decision-index-results.
• Leaderboard. The board ranks by the Full score, which comes from the maintainers' private tests: 60% tests of the same skills as the public benchmarks and 40% decision tasks from new domains. The public score is not part of it.

The index and area scores are chance-corrected: 0 is random guessing and 100 is perfect. Below, the score is in each benchmark's own metric and the skill is chance-corrected.

All 43 benchmarks

Calibration

The temperatures were fitted on the calibration split (17,753 questions), which was not used for training or for choosing the weights.

Latency

Per Decision Index request. In-process wall time of one request, CUDA-synchronized, eager, one request at a time, over the 140,620 requests of the 0.3 run, with one engine per NVIDIA RTX PRO 6000 Blackwell Server Edition: median 219 ms, mean 913 ms, 80th percentile 657 ms, 95th percentile 1,507 ms. This is not the maintainers' measurement. The leaderboard measures latency separately, also on one NVIDIA RTX PRO 6000, and lists a median of 232 ms, a mean of 663 ms and a 95th percentile of 1,520 ms.

Training

• Recipe.
  • Full fine-tune of every linear projection of the 60 decoder layers (29,286,727,680 parameters); the token embeddings (tied with the output layer) and every norm stay frozen. The 10 global attention layers were bidirectional during training, as in serving.
  • Loss: cross-entropy on the option-label logits + 0.3 × the multi-class Brier score.
  • AdamW, lr 2e-6, betas (0.9, 0.95), weight decay 0.01 on weight matrices, gradient clipping at 1.0, 5% linear warm-up then cosine decay to 0.1×, 1 epoch, 128 records per step.
  • bf16 compute with fp32 master weights, gradient reduction and optimizer state (AdamW moments in CPU memory); FSDP2 with one unit per decoder layer; gradient checkpointing.
  • A record's questions share one pass over its state under the attention rule above, in packs of up to 16,384 tokens. Longer records are dropped, never truncated (none were).
  • Permuted choice options, "none of the above" augmentation (as the answer, as a wrong option and in minimal pairs) and irrelevant distractor options.
  • 5,544 steps in two stages on two NVIDIA B300 GPUs: steps 1–500 (4.4 h), a check of the step-500 weights, then steps 501–5,544 (44.13 h) resumed with the optimizer state; about 97 GPU-hours.
• Release weights. Weight snapshots were kept at steps 500, 1,386, 2,772, 4,158 and 5,544. The released weights are 0.3 × the backbone + 0.7 × the step-5,544 weights, chosen among the snapshots, their average and their blends with the backbone by a selection rule fixed before training. The two-order reading was chosen by the same rule. The Decision Index benchmarks were never used to choose.
• Held-out monitor. On a fixed 6,000-question held-out sample, the fine-tuned weights reached accuracy 0.8925 and NLL 0.2797 at step 5,544.
• Data. 709,604 training records (957,775 questions), with 12,395 records for calibration and 12,902 held out.
  • Sources. Public datasets and generated decision records.
  • Not published. The training files are not part of this release; the SHA-256 of each file is in training_config.json.

Limitations

• score questions. The training data has only choice and noul questions. score questions are answered by the same readout with the levels as options, but the model was not trained on them.
• More than 255 options. The readout has 255 option labels; a choice question with more options is refused.

Files

License

The weights are released under Apache-2.0 (see LICENSE), as is the code at github.com/sthanika-ai/Sieve. Use of the base model is governed by Gemma's terms, including Google's prohibited-use policy.