Sieve-31B-v2

Sieve-31B-v2 is a decision model from the Sieve family. You give it a piece of text or JSON (the state) and typed questions, and it returns a calibrated probability for every option of every question. It reads the state once, scores the options directly, and never generates text.

It is a full fine-tune of google/gemma-4-31B-it (revision 842da37), read at one answer slot per question, like Sieve-31B. Unlike Sieve-31B, it uses bidirectional attention in its 10 global attention layers: they read the whole prompt in both directions, while the other 50 layers stay causal. The code is at github.com/sthanika-ai/Sieve.

Summary

How it works

• Prompt. Each question is one user turn in Gemma's own chat template, with thinking off, followed by an Answer: ( slot. Options keep the order the request gives them.
• Score. At the slot, the final hidden state is multiplied by the output-layer rows of the option label tokens (A, B, …, up to 255 options). A softmax over the question's own options at the fixed temperature T = 8.5495 gives the probabilities.
• Bidirectional attention. See the next section.
• Weights. All 60 decoder layers are stored in full as a PEFT adapter (modules_to_save), next to a LoRA on the embeddings that is exactly zero, as in Sieve-27B-v2. sieve 0.3.3 and later copy the layers into the backbone in place, so the model needs about the backbone's memory.
• Never. It never truncates, drops or reorders options, tunes prompts per benchmark, or sees gold labels.
• Declared limits. Up to 255 options per question and 32,768 tokens per prompt row (state plus one question). Larger requests are refused as Unsupported. No public request reached either limit.
• Out of memory. The forward chunk is halved and the request retried. A single row that still does not fit raises an ordinary error, so a resumed run retries it.

Bidirectional attention

Gemma-4-31B-it interleaves 10 global (full-attention) layers with 50 sliding-window layers (1,024-token window). A text-generation model has to be causal: each token only sees the tokens before it. A decision model generates nothing, so it does not need that restriction.

• What changes. In Sieve-31B-v2 the 10 global layers are bidirectional over the prompt: every token attends to every other prompt token, before and after it. The 50 sliding-window layers stay causal, as in the base model.
• Why. The state comes first in the prompt and the question and its options come after it. With bidirectional global layers, the model reads the state in the light of the question and options it has to decide between, instead of reading the state before it has seen what is asked.
• Trained and served the same way. The fine-tune was trained with this attention, and sieve serves it the same way. training_config.json records the mode ("attention": "bidir_global"), and sieve switches the global layers to bidirectional when it loads the model. Sieve-31B and other releases without this field load causal, as before.
• Cost. None. A single request passes no mask to the global layers, so they run the same fused attention kernel as a causal pass. A padded batch passes a key-padding mask, so padding is never attended to.
• Isolation. Each question is its own row (state plus one question), so questions never see each other, in either direction.

Use

pip install torch==2.8.0 --index-url https://download.pytorch.org/whl/cu128
pip install transformers==5.17.0 peft==0.21.0 accelerate==1.15.0 huggingface_hub==1.32.0 jinja2 httpx pydantic
pip install --no-deps decider-ai==1.6.0 "sieve-decisions @ git+https://github.com/sthanika-ai/Sieve"
pip install --no-deps "decision-index @ git+https://github.com/apolinario/decision-index@62d2f51"

from sieve import load_sieve, decide

m = load_sieve("sthanika-ai/Sieve-31B-v2")
decide(m, {"subject": "Duplicate charge on invoice 4411", "body": "Billed twice. Refund today or we cancel."},
       {"department": {"type": "choice", "instructions": "Which team handles this?",
                       "criteria": {"billing": "invoices, payments, refunds", "technical": "bugs and outages"}},
        "churn_risk": {"type": "noul", "instructions": "Does the customer threaten to cancel?"}})

• Loading. Load it with sieve 0.3.3 or later, which loads the full fine-tune in place and switches the global layers to bidirectional attention; the adapter alone is not a text-generation model. The backbone is downloaded at its pinned revision on first use.
• Requirements. The decider-ai package (1.6.0) builds the prompts; install it without its dependencies, as above.
• Server. sieve-serve --model sthanika-ai/Sieve-31B-v2.
• Decision Index. python -m decision_index pipeline --engine sieve.decision_index_engine:SieveEngine --option model=sthanika-ai/Sieve-31B-v2 --out runs/Sieve-31B-v2.
• Smaller GPUs. --option split='{"0": "40GiB", "1": "40GiB"}' spreads the model over two GPUs.

Results

Decision Index

Sieve-31B-v2 scores 69.97 (raw index 76.95, breadth 69.44) on the public 0.3 suite of the Decision Index, on our own run with the official kit at 62d2f51 (0.3) and its scorer.

• Coverage. All 140,178 scoreable requests over 43 benchmarks were answered, with no truncation, no unsupported request and no error. The index averages 37 of the benchmarks.
• Results. sthanika-ai/Sieve-31B-v2-decision-index-results.
• Not on the leaderboard yet. The score is self-reported.

The index and area scores are chance-corrected: 0 is random guessing and 100 is perfect. Below, the score is in each benchmark's own metric and the skill is chance-corrected.

All 43 benchmarks

Latency

Measured with this engine on 1× RTX PRO 6000 Blackwell Server Edition: single process, eager, one request at a time, over 400 random public Decision Index 0.3 requests (the same 400 as Sieve-31B's check). Median 110 ms, mean 437 ms, 80th percentile 462 ms, 95th percentile 1,763 ms (Sieve-31B on the same rows: median 163 ms, mean 644 ms).

Training

• Method. Full fine-tune of the 60 text decoder layers in fp32 master weights with bf16 compute; the token embeddings, final norm and output layer stay the backbone's. The 10 global attention layers were bidirectional during training, as in serving; the 50 sliding-window layers were causal.
• Objective. Cross-entropy at T = 8.5495 on the option-label logits at the answer slot + 0.3 × multi-class Brier score.
• Optimisation. 8-bit AdamW, peak learning rate 4e-6 with a 5% warm-up and cosine decay to 0.1×.
• Weights. The released weights are the uniform average of three checkpoints of one run, taken at 25%, 50% and 75% of its schedule.
• Hardware. 2× NVIDIA B300.
• Settings. The settings are in training_config.json.

Files

License

The weights are released under Apache-2.0, as is the code at github.com/sthanika-ai/Sieve. Use of the base model is governed by Gemma's terms, including Google's prohibited-use policy.