This repo contains specialized MoE-quants for XiaomiMiMo/MiMo-V2.6-Flash-MOPD. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization. To that end, the quantization type default is kept in high quality and the FFN UP + FFN GATE tensors are quanted down along with the FFN DOWN tensors.
The MXFP4 quant is the "full quality" version, as the model has MXFP4 experts.
The BPW quants were produced using ed's bpw-size PR. The FFNs are the primarily quantized feature, rest of the model remains in Q8_0 / Q6_K
| Quant | Size | Mixture | PPL | 1-(Mean PPL(Q)/PPL(base)) | KLD |
|---|---|---|---|---|---|
| MXFP4 | 162.89 GiB (4.52 BPW) | BF16 / MXFP4 | 5.147382 ± 0.030566 | +0.0753% | -0.000000 ± 0.000000 |
| BPW3.5 | 126.18 GiB (3.50 BPW) | Q8_0 / varies | 5.248009 ± 0.031054 | +2.0317% | 0.134408 ± 0.000692 |
| BPW3.0 | 108.16 GiB (3.00 BPW) | Q8_0 / varies | 5.425520 ± 0.032234 | +5.4829% | 0.173319 ± 0.000870 |
| BPW2.5 | 90.09 GiB (2.50 BPW) | Q6_K / varies | 5.684465 ± 0.034102 | +10.5173% | 0.226887 ± 0.001093 |
| BPW2.0 | 72.07 GiB (2.00 BPW) | Q6_K / varies | 6.544785 ± 0.040620 | +27.2436% | 0.365580 ± 0.001645 |
BPW2.0/MiMo-V2.6-Flash-MOPD-BPW2.0-00002-of-00003.gguf 50 GBBPW2.0/MiMo-V2.6-Flash-MOPD-BPW2.0-00003-of-00003.gguf 27 GBBPW2.0/MiMo-V2.6-Flash-MOPD-BPW2.0-00001-of-00003.gguf 6 MBBPW2.5/MiMo-V2.6-Flash-MOPD-BPW2.5-00002-of-00003.gguf 50 GBBPW2.5/MiMo-V2.6-Flash-MOPD-BPW2.5-00003-of-00003.gguf 47 GBBPW2.5/MiMo-V2.6-Flash-MOPD-BPW2.5-00001-of-00003.gguf 6 MBBPW3.0/MiMo-V2.6-Flash-MOPD-BPW3.0-00003-of-00004.gguf 50 GBBPW3.0/MiMo-V2.6-Flash-MOPD-BPW3.0-00002-of-00004.gguf 50 GBBPW3.0/MiMo-V2.6-Flash-MOPD-BPW3.0-00004-of-00004.gguf 17 GBBPW3.0/MiMo-V2.6-Flash-MOPD-BPW3.0-00001-of-00004.gguf 6 MBBPW3.5/MiMo-V2.6-Flash-MOPD-BPW3.5-00003-of-00004.gguf 50 GBBPW3.5/MiMo-V2.6-Flash-MOPD-BPW3.5-00002-of-00004.gguf 49 GBBPW3.5/MiMo-V2.6-Flash-MOPD-BPW3.5-00004-of-00004.gguf 36 GBBPW3.5/MiMo-V2.6-Flash-MOPD-BPW3.5-00001-of-00004.gguf 6 MBMXFP4/MiMo-V2.6-Flash-MOPD-MXFP4-00003-of-00005.gguf 49 GBMXFP4/MiMo-V2.6-Flash-MOPD-MXFP4-00004-of-00005.gguf 49 GBMXFP4/MiMo-V2.6-Flash-MOPD-MXFP4-00002-of-00005.gguf 49 GBMXFP4/MiMo-V2.6-Flash-MOPD-MXFP4-00005-of-00005.gguf 28 GBMXFP4/MiMo-V2.6-Flash-MOPD-MXFP4-00001-of-00005.gguf 6 MBHave the files? Submit a torrent.
Submit a torrent