Qwen3.8-27B-oQ4e-fp16-mtp
[!NOTE] Updated 2026-10-29: Re-quantized with the improved oQe quantization in oMLX 0.7.3. Please re-download if you have an earlier copy.
This model was quantized using oQ (oMLX v0.7.3) mixed-precision quantization with improved oQe (jundot/omlx#4930).
A mixed-precision MLX checkpoint of Qwen/Qwen3.8-27B, a 27B dense vision-language model built on the Qwen3.5 architecture, with a hybrid layout of Gated DeltaNet and Gated Attention layers, image and video understanding and a 262,144-token native context. It includes the language model, the vision encoder and the multi-token prediction (MTP) head.
This FP16 build stores the quantization scales and biases and the remaining unquantized tensors in FP16 instead of BF16, and keeps the vision encoder in FP32.
The effective average is 4.840 bits/weight for the language backbone and 5.310 bits/weight for the complete checkpoint.
Quantization and Bit Distribution
oQ4e allocates bits by measured layer sensitivity, then rounds each group with an importance matrix (128 samples x 512 tokens) and a weighted least-squares refit of its scale and bias.
| Storage format | Logical weights | Share of total | Tensor storage | Effective bits/weight |
|---|---|---|---|---|
| Affine 4-bit, group 64 | 19.393B | 69.807% | 10.160 GiB | 4.50 |
| Affine 5-bit, group 64 | 6.579B | 23.680% | 4.212 GiB | 5.50 |
| Affine 6-bit, group 64 | 1.282B | 4.615% | 0.970 GiB | 6.50 |
| Affine 8-bit, group 64 | 0.011B | 0.041% | 0.011 GiB | 8.50 |
| FP16 | 0.055B | 0.198% | 0.103 GiB | 16.00 |
| FP32 | 0.461B | 1.658% | 1.716 GiB | 32.00 |
| Total | 27.781B | 100% | 17.172 GiB | 5.310 |
Storage Breakdown
| Component | Logical weights | Storage |
|---|---|---|
| Language backbone | 26.896B | 16.272 GB / 15.155 GiB |
| Vision encoder | 0.461B | 1.843 GB / 1.716 GiB |
| MTP head | 0.425B | 0.323 GB / 0.301 GiB |
| Total | 27.781B | 18.439 GB / 17.172 GiB |
License
Apache-2.0, following the source model. See LICENSE.