Ling-3.1-flash-1019-oQ2e-mtp
This model was quantized using oQ (oMLX v0.7.3rc1) mixed-precision quantization.
A mixed-precision MLX checkpoint of inclusionAI/Ling-3.1-flash-1019, an upgraded version of Ling-3.1-flash with improved reasoning and agentic capabilities. It is a hybrid-linear mixture-of-experts model with 560B total and 25B active parameters, built from 63 KDA linear-attention layers and 9 gated MLA layers, with 512 routed experts of which 8 are active per token. It includes the language model and the MTP (NextN) layer, which oMLX runs as Lightning MTP speculative decoding.
The effective average is 2.701 bits/weight for the complete checkpoint.
Quantization and Bit Distribution
oQ2e allocates bits by measured layer sensitivity, then rounds each group with an importance matrix (256 samples x 512 tokens, collected layer by layer from the original FP8 weights) and a weighted least-squares refit of its scale and bias.
| Storage format | Logical weights | Share of total | Tensor storage | Effective bits/weight |
|---|---|---|---|---|
| Affine 2-bit, group 64 | 547.608B | 95.716% | 159.375 GiB | 2.50 |
| Affine 4-bit, group 64 | 8.053B | 1.408% | 4.219 GiB | 4.50 |
| Affine 8-bit, group 128 | 1.085B | 0.190% | 1.042 GiB | 8.25 |
| Affine 8-bit, group 64 | 15.333B | 2.680% | 15.173 GiB | 8.50 |
| BF16 | 0.041B | 0.007% | 0.076 GiB | 16.00 |
| FP32 | 0.000B | 0.000% | 0.000 GiB | 32.00 |
| Total | 572.121B | 100% | 179.885 GiB | 2.701 |
Storage Breakdown
| Component | Logical weights | Storage |
|---|---|---|
| Language backbone | 563.946B | 188.460 GB / 175.517 GiB |
| MTP head | 8.174B | 4.690 GB / 4.367 GiB |
| Total | 572.121B | 193.150 GB / 179.885 GiB |
License
MIT, following the source model.