Decision 2.0 Kai-0.6B, MLX 8-bit

An 8-bit MLX conversion of vllm-sr/Decision-2.0-Kai-0.6B at revision cd49ea3813fd8ba0928a9a23ef6c9a0f2f0cd764.

• model.safetensors, config.json: the Qwen3 backbone, renamed to the mlx_lm qwen3 layout and quantized with mlx_lm 0.32.0 (affine, 8 bits, group size 64).
• decision/: the original decision head (decision_head.safetensors), tokenizer, decision_config.json, MODEL_MANIFEST.json and LICENSE, unchanged.

It is the local classifier of Mail Assistant, which runs it on mlx-swift 0.32.3.

Parity

Measured on 95 private emails with one Choice question of 19 options (prompts of 1.4k to 3.7k tokens, no truncation):

• Same answer as the official PyTorch FP32 runtime on 93 of 95 emails.
• Same answers as the Python MLX runner with the same weights on 95 of 95, with a maximum confidence difference of 4e-6.

At 4 bits the same backbone agreed with the official runtime on only 72 of 95, so this repository stays at 8 bits.

License

Apache-2.0, like the original model. The original LICENSE is in decision/.