Decision-2.0-Kai-0.6B (ONNX)

ONNX conversion of vllm-sr/Decision-2.0-Kai-0.6B, a Decision 2.0 model from vLLM Semantic Router: give it a state and typed questions (choice, noul, score) and it returns a probability for every option. For the model, its training and evaluation, see the original card.

It runs in the browser on WebGPU with open-jev (model: "decision2-kai-0.6b"), or with ONNX Runtime anywhere.

How it was made

Decision 2.0 is a full fine-tune of Qwen3-0.6B-Base with a candidate head on the final hidden states. Qwen3 is a plain transformer, so the backbone and the head were exported together with torch.onnx (fp32, eager attention), then weight-quantized with ONNX Runtime's MatMulNBits quantizer (embeddings 4-bit, matmuls 8-bit).

input_ids [B, L], attention_mask [B, L], answer_pos [B], option_pos [B, K]  ->  logits [B, K]

Build one prompt per question as the package's decision_model.segments / encode do (Context: … Task type: … Question: … Options:, one <option> segment per option holding {"description": …, "key": …} as sorted, compact JSON, then the Decision: suffix), tokenizing each segment separately; option_pos is the last token of each option segment and answer_pos the last token. Add decision2.score_bias[L] from config.json to the logits of an L-level score question (Kai ships offsets for 5 levels), then softmax (temperature 1).

Scripts and reference cases are in conversion/.

Files

q8 here is 8-bit block quantization (MatMulNBits, bits=8), not the usual dynamic int8 export; it uses the q8 slot because Transformers.js has no q8f16.

Checks

Against the packaged runtime (decision2.api.Decision2, exact path, CPU fp32), on 11 questions over 6 states: largest difference in any option's probability and how many top answers changed.

open-jev in Node reproduces the Python run of the same graph exactly (same token ids, option positions and score offsets).

In the browser

On a MacBook Pro (M3 Pro, Chrome, WebGPU), on the 400 states × 5 questions of typed-decisions (test split; top answer = annotator label, as in the open-jev demo's benchmark):

License

Apache-2.0, as the original model.