OD-1 Base (4B)

OD-1 is an open System One decision model: give it a state (text or JSON) and typed questions (choice, noul = yes/no, score = ordinal) and it returns every answer with full probability distributions (plus an explicit NOT_ANSWERABLE outcome), in the Jev / TypeSafe /v1/systemone shape.

Qwen3.5-4B, fully fine-tuned with the OD-1 recipe; the checkpoint was selected on out-of-domain validation accuracy (later checkpoints scored higher in-domain but lost zero-shot ability).

When to use it

The most accurate zero-shot OD-1. Use the built-in hand-off to answer most requests at Nano speed.

Results on held-out test sets

Same fixed samples for every system (≤ 2,000 decisions per set, seed 0; typed-decisions: all 2,000 test decisions). Baselines answer each question as one choice over its options (2–24 options; n/a = more options than their interface allows). Jev 1.13 = jev-1.13.0, measured 2026-09-25. Accuracy:

On the typed-decisions test split, scored with the benchmark card's formulas (KL from the soft gold and Brier verified against the card's reference rows): accuracy 0.582, KL 0.496, Brier 0.241 (generalist, zero-shot). For reference the card lists TypeSafe Jev 1.13 at 0.727 / 1.442 / 0.148 and meraGPT Decider 1 at 0.768 / 0.096 / 0.052 (both generalists).

Built-in hand-off (self-contained)

This repo bundles the matching Nano model in nano/ and a fitted hand-off in cascade.json:

• two-model: the Nano answers first; the request goes to this model when the Nano's lowest per-question confidence < 0.5.
• self-exit: this model answers at its layer-8 exit first and runs its full depth when that exit's lowest confidence < 0.99.
• Default: two-model (recommended on validation data; thresholds fitted on validation data only).
• Validation: nano alone 0.737, this model alone 0.761, two-model hand-off 0.766 with 0.283 of requests escalated.
• Validation, self-exit: exit alone 0.589; at tau 0.99 accuracy 0.760 with 0.997 escalated (an early exit this shallow rarely clears the threshold, so self-exit mostly costs time).
• Test (two-model): typed-decisions 0.582, AG News 0.802, Emotion 0.560, SST-5 0.482, Banking77 0.541, MASSIVE-en 0.679, BFCL native 0.945, BFCL 24 options 0.936, BFCL 100 options 0.868, BFCL 1,000 options 0.474, BFCL irrelevance 0.579, Adversarial (ours) 0.892.
• Test (self-exit): typed-decisions 0.582, AG News 0.843, Emotion 0.559, SST-5 0.474, Banking77 0.482, MASSIVE-en 0.694, BFCL native 0.942, BFCL 24 options 0.938, BFCL 100 options 0.851, BFCL 1,000 options 0.439, BFCL irrelevance 0.586, Adversarial (ours) 0.887.

from od1.cascade import OD1Cascade
m = OD1Cascade.load(path)                  # default mode; or mode="two-model" / "self-exit"
m.enable_cuda_graphs()
answers = m.answer(state, questions)   # each answer says which model answered

Speed (one H100, bf16, batch-1 requests, CUDA graphs)

Limitations

• On unfamiliar tasks accuracy is well below the best closed systems (see the table); NOT_ANSWERABLE is only returned above the fitted threshold in serving.json.

• English only. Baselines were called through a choice-wrapped interface (Laya's typed-decisions checkpoint also natively), which may understate them. No frontier-LLM reference was run.

• Scores on typed-decisions measure agreement with its labelling teacher (see the dataset card).

Usage

pip install torch "transformers>=5.17.0" huggingface_hub

from huggingface_hub import snapshot_download
path = snapshot_download('mvbalaji/od1-base')
import sys; sys.path.insert(0, path)
from od1.model import OD1Model
m = OD1Model.load(path, dtype='bf16', device='cuda')   # applies the fitted serving.json
m.enable_cuda_graphs()
answers = m.answer(state, {'route': {'type': 'choice', 'instructions': 'Which team?', 'criteria': {'billing': 'payments', 'tech': 'bugs'}}})

Smoke test of example.py for this checkpoint: {"route": {"answer": "billing", "confidence": 0.994}, "refund": {"answer": 0.9936571213287037, "confidence": 0.99}, "urgency": {"answer": 2, "confidence": 0.461}}

Requirements and speed

The Qwen3.5 backbone mixes full attention with gated delta-rule (linear attention) layers. Install the fast kernels, otherwise transformers silently falls back to a slow pure-PyTorch implementation of those layers and a one-question request takes hundreds of milliseconds instead of a few:

pip install "transformers>=5.17.0" flash-linear-attention==0.5.2
pip install causal-conv1d==1.7.0 --no-build-isolation   # compiles against your torch/CUDA (needs nvcc)

Check: import fla, causal_conv1d must succeed; when they are missing, transformers logs "falling back to its reference PyTorch implementation". Then enable CUDA graphs and warm up before timing:

m.enable_cuda_graphs()
for _ in range(5):                       # warm-up: kernel autotuning + graph capture per input shape
    m.answer(state, questions)
torch.cuda.synchronize(); t0 = time.perf_counter()
answers = m.answer(state, questions)
torch.cuda.synchronize(); print((time.perf_counter() - t0) * 1000, "ms")

Tested with torch 2.14.0+cu130, transformers 5.17.0, flash-linear-attention 0.5.2, causal-conv1d 1.7.0 on one NVIDIA H100 80GB HBM3 (bf16). Expected on that setup for a short one-question request: about 35 ms without CUDA graphs and about 6 ms with them (see the speed table); other GPUs differ. CPU inference works but is slow.

Training data and licenses

Openly licensed datasets (see sources.csv: dataset, revision, license, split, counts), project-generated synthetic policy checks and teacher-verified structured JSON workflows (written by Qwen3.5-27B-FP8, which is never a compared system, without access to any benchmark data), with teacher distillation labels. Held-out evaluation sets were never trained on (13-gram deduplication against every test set).

Citation

Paper in preparation (OpenDecide-1).