decider-2b (weightless ONNX, fp32 compute)
Weightless ONNX export of
Mapika/decider-2b v12
(Qwen3.5-2B-Base, Apache-2.0): one forward pass, typed decisions with
calibrated probabilities — no generation. Same bytes Ollaya serves as
decider:2b, staged here self-contained.
model.onnx(7.6 MB graph) +model.safetensors(3.76 GB BF16, upstream bytes vendored) +tokenizer.json+decision.json+calibration.json.- License: Apache-2.0.
- Graph built algebraically
(ollaya
convert/ollaya_convert: every checkpoint tensor maps to an external BF16 reference + Cast to fp32), not torch-traced: the Gated-DeltaNet chunked rule rejects symbolic sequence dims, so dynamo export hangs. Provenance + hashes:manifest.json.
Contract (decider-slots-v1)
Inputs: input_ids int64 [rows, seq] (seq a multiple of 64,
right-pad any id), slot_pos int64 [rows] (answer-slot positions).
Output: label_logits float32 [rows, 255] — tied-LM-head rows for
the 255 option labels; use the first k. Score questions run one
isolated yes/no row per level (decision.json has the full layout).
Serve
import onnxruntime as ort
so = ort.SessionOptions()
so.enable_mem_reuse = False # required on CUDA (see below)
sess = ort.InferenceSession("model.onnx", so, providers=["CUDAExecutionProvider"])
CUDA requires enable_mem_reuse = False: the CUDA memory planner
otherwise corrupts the int64 shape values the chunk-scan Reshapes read
(requested shape {8,16,0,64} — documented in Ollaya's runner). CPU
keeps defaults. Temperatures come from calibration.json
(choice 2.465 / score 1.0 / noul 0.05 — the 0.05 is deliberate,
for JevBench's 0.2–0.8 band rule; calibrated yes/no needs 3.185,
see the upstream card).
Measured (RTX 3090 + Ryzen 9 5950X)
| Check | Result |
|---|---|
| Graph | opset 18, 7,976 nodes, zero non-standard ops, 0 unused tensors |
| torch↔ORT parity | max|Δp| 1.0e-4 CPU / 1.9e-4 CUDA (choice + noul + score, winners agree 3/3) |
| CPU | ~1.5–2 s/5-row batch — slow-tier only |
| CUDA | ~373 ms/5-row batch (~75 ms/row; Ollaya: 91 ms/5q on 5090) |
Full bakeoff notes: see the embroider repo,
docs/typed-decision-plan.md §9c (dynamo rejection, CUDA workaround,
int8 status).
q8
decider-2b q8: 186 decoder linears to int8 MatMulNBits (block 32, fp32 scales); slot head (dynamic-B), embeddings, norms untouched (still served from model.safetensors). Gate: worst 0.0020, 0 flips, 10/10 golden agree (incl. 0.05-temp noul). Parity: worst 0.0020, 0 flips (10/10 golden agree).