respanai/span-02

Read the announcement: Introducing span-2 →

Span-02 is a frontier decision model. Send it a state (text or JSON, plus up to 4 images) and typed questions; it returns a calibrated probability distribution for each answer. No free text, no parsing.

• Typed answers: choice, yes/no (noul), score on a scale, or number.
• Fast: most requests answer in tens of milliseconds. When unsure, the model thinks briefly first.
• Efficient: Gemma 4 26B-A4B fine-tune (~4B active parameters), NVFP4/FP8 (16.4 GB), served by its own Rust/CUDA engine included here.
• One GPU: RTX PRO 6000 or RTX 5090 (Blackwell).

Decision Index

_{Official Decision Index: span-02 is on the latency–score frontier at 33 ms median.}

Quickstart

The weights run only with the included engine (engine/), not with transformers or vLLM.

Needs an NVIDIA driver with CUDA 12.9 support and the NVIDIA container toolkit:

hf download respanai/span-02 --local-dir span-02 && cd span-02
docker build -t span-02 .
docker run --gpus all -p 8000:8000 -v "$PWD":/model:ro -e DLLM_MODEL_DIR=/model -v span02-data:/data span-02

Ready in about 20 s, when GET /health returns {"ok": true, "model": "span-02"}. Without Docker, build the engine with cargo build --release -p dllm-server in engine/ (CUDA 12.8+) and run bash launch.sh.

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": {"customer": "My package arrived damaged and nobody answers my emails. I want a refund now."},
  "questions": {
    "dept":   {"type": "choice", "instructions": "Which team should handle this?",
               "criteria": {"returns": "Exchanges, wrong or damaged items",
                            "shipping": "Delivery status, delays, lost packages",
                            "billing": "Charges, invoices, payment problems"}},
    "refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"}
  }
}'

{"model": "span-02",
 "answers": {
   "dept":   {"type": "choice", "choice": "returns", "confidence": 0.945,
              "probabilities": {"returns": 0.963, "shipping": 0.024, "billing": 0.013}},
   "refund": {"type": "noul", "noul": 0.99999}},
 "usage": {"input_tokens": 410, "reasoning_tokens": 0, "output_tokens": 0, "latency_ms": 31.0}}

(Values are illustrative.) The API follows TypeSafe's System One API, so its clients work unchanged.

How it works

1. Read the answer directly: one forward pass per question reads the probabilities of the option labels. The shared state is computed once per request.
2. Think if unsure: if the top answer is below 0.8, the model writes a short thought, reads again, and averages the two distributions.
3. Stay on time: thoughts have a time budget (700 ms by default) and use speculative decoding.

API

POST /v1/systemone answers a request; GET /health reports readiness.

The response has answers (plus reasoning for questions that thought) and usage (input, reasoning and output tokens, server latency). Errors are {"error": {"type", "message"}} with the usual HTTP statuses; 529 means busy, retry shortly.

Full API reference

Request

• mode: auto (default) thinks only when unsure; on always thinks; off never thinks.
• budget_ms: the thinking time limit for this request; 0 means no limit. If omitted, on thinks until the thought ends (at most 4,096 tokens) and auto uses the server default (--budget-ms, 700).
• trace: return each thought's text.
• Bodies can be JSON, multipart/form-data (a request part plus one file part per image, referenced as {"part": name}), or compressed with Content-Encoding: zstd or gzip. Unknown top-level fields are ignored.

Question types

• choice confidence = (p_max − 1/n) / (1 − 1/n): 0 for a uniform guess, 1 for certainty.
• score confidence = 1 − (expected distance from the most likely level) / (the same for a uniform guess).
• number confidence = exp(mean log-probability of the answer tokens). Number questions think only with mode: "on", because the think gate is trained on option questions.

Response

• answers[id]: the typed answer. If the question thought, it also has reasoning: {"tokens": N} (plus trace when requested).
• usage: input_tokens (the prompt tokens of every question; the shared state counts once per question and each image its image tokens + 2), reasoning_tokens (thought tokens), output_tokens (number-answer tokens; other answers are read in one pass and generate none), latency_ms (server time).
• image_tokens: total image tokens, present only when the request has images.

Images

The model was trained with all images (up to 4) before the text, referred to as "Image 1", "Image 2", and so on. The server shows images first, then assets in the order sent, and replaces every mention of an asset id in the state, instructions and criteria with that image's label. For example {"state": {"photo": "a3f1c2e8-found"}, "assets": {"a3f1c2e8-found": ..., "77b0d4a1-poster": ...}} reaches the model as {"photo": "Image 1"}, and "the dog in this picture: 77b0d4a1-poster" becomes "the dog in this picture: Image 2". Assets that are never mentioned are still shown. Images are PNG or JPEG, at most 16 MiB each, and are resized to the image budget (280 tokens per image by default, 560 in the Decision Index settings).

Errors

Errors look like {"error": {"type": "...", "message": "..."}}.

Performance

On an RTX 5090 the engine is 1.3 to 7× faster than vLLM without thinking.

Requirements

• NVIDIA Blackwell GPU, compute capability 12.0: RTX PRO 6000 (96 GB) or RTX 5090 (32 GB).
• Driver with CUDA 12.9 support (R575+). About 28 GB of GPU memory by default.
• Quantization: MoE experts NVFP4; attention, dense MLP and LM head FP8; vision tower and KV cache bf16.
• Limits: 131,072-token context per question, 4 images, 64 MB body.

Decision Index settings

The Decision Index submission runs launch.sh as shipped:

--max-len 131072 --budget-ms 0 --tau 0.8 --cap 1024 --think-ms 750 --think-cost think_cost.json
--image-soft-tokens 560 --serial --no-prefix-cache --spec-max-branches 16 --pages 12000 --kv-gb 24

• Deterministic thinking budget: --budget-ms 0 --think-ms 750 --think-cost think_cost.json. The wall clock is never read. Instead, a fixed cost model turns a 750 ms target into a thought-token cap that depends only on the request (prompt length, images, number of thinking questions), so the result does not depend on the machine or its load. A short single question gets about 330 thought tokens; requests with many thinking questions or long prompts get fewer. --cap 1024 is the hard limit.
• One request at a time, no prefix cache: --serial --no-prefix-cache, so an answer does not depend on other requests. Requests where several questions think at once can still vary very slightly in their post-thought probabilities between runs.
• Image budget: --image-soft-tokens 560, about 1.3 MP with the aspect ratio kept.
• KV cache: --pages 12000 --kv-gb 24, sized so every Decision Index request fits on its own.

License

The weights derive from Google's Gemma 4 (google/gemma-4-26B-A4B-it) under the Gemma 4 license (Apache 2.0). The engine is Apache-2.0 and credits the projects it ports from (vLLM, SGLang, FlashInfer and others) in NOTICE.

Citation

@misc{li2026span02,
  title        = {span-02: a fast multimodal decision model},
  author       = {Li, Harvey},
  organization = {Respan},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/respanai/span-02}},
  note         = {Blog post: \url{https://www.respan.ai/blog/introducing-span-2}}
}

Contact: Harvey Li (harvey@respan.ai), Respan.