d3-oQ8e
This model was quantized using oQ (oMLX v0.7.1) mixed-precision quantization with Improved oQe Quantization (jundot/omlx#4385).
A mixed-precision MLX checkpoint of vllm-sr/d3, the 27B multimodal decision model of Decision 3.0 from vLLM Semantic Router, built on Qwen/Qwen3.8-27B. It reads text or JSON with optional images and videos and returns a probability for every option of choice, yes/no and score questions without generating text. It includes the language backbone, the vision encoder and the code readout.
The code readout (readout.safetensors, FP32) and decision_config.json are copied unchanged from the source. oMLX serves this checkpoint through POST /v1/systemone; Decision 3.0 support (jundot/omlx#4470) is newer than oMLX v0.7.1.dev1.
The effective average is 8.634 bits/weight for the complete checkpoint.
Quantization and Bit Distribution
oQ8e allocates bits by measured layer sensitivity, then rounds each group with an importance matrix (128 samples x 512 tokens) and a weighted least-squares refit of its scale and bias.
| Storage format | Logical weights | Share of total | Tensor storage | Effective bits/weight |
|---|---|---|---|---|
| Affine 8-bit, group 64 | 25.622B | 98.219% | 25.354 GiB | 8.50 |
| BF16 | 0.463B | 1.776% | 0.863 GiB | 16.00 |
| FP32 | 0.001B | 0.005% | 0.005 GiB | 32.00 |
| Total | 26.087B | 100% | 26.222 GiB | 8.634 |
Storage Breakdown
| Component | Logical weights | Storage |
|---|---|---|
| Language backbone | 25.625B | 27.229 GB / 25.359 GiB |
| Vision encoder | 0.461B | 0.921 GB / 0.858 GiB |
| Code readout | 0.001B | 0.005 GB / 0.005 GiB |
| Total | 26.087B | 28.155 GB / 26.222 GiB |
License
Apache-2.0, following the source model. See LICENSE and NOTICE.