A compressed XiaomiMiMo/MiMo-V2.6-Flash-RL (309B / 15B-active MoE) in stock mlx-lm format, sized so the text model loads on a 128 GB Apple Silicon machine (M3 Max / M4 Max / Ultra). Text model on disk: 87.4 GB (128 experts/layer, experts average 4.25 bits/weight). Bundled MTP / DFlash / vision / audio weights: 7.3 GB (not loaded for text inference).
The official checkpoint's experts are natively MXFP4 (Xiaomi trained them with MXFP4 QAT; there is no BF16 release), so 4-bit MLX conversions are lossless and anything smaller is a real re-quantization. This model was produced by a calibrated, layer-wise pipeline (PyTorch, one H200):
attention_value_scale is folded into v_proj (the mlx-lm class has none).mtp/), DFlash drafter (dflash/), vision & audio encoders (omnimodal/, audio_tokenizer/) are carried over from Vontra's conversion unchanged; the checkpoint layout is theirs, so whatever loads that model loads this one.Calibration: 256 sequences × 2048 tokens from evol-codealpaca, Mixture-of-Thoughts, SWE-smith trajectories, glaive function calling and UltraChat, rendered with the model's chat template. Held-out evaluation uses disjoint samples from the same mix.
| Metric (held-out agentic/coding mix, 31×2048 tokens) | Original (MXFP4/FP8) | This model |
|---|---|---|
| Perplexity | 9.271 | 6.996 |
| KL(original ‖ this), mean per token | 0 | 1.2959 |
| Top-1 next-token agreement with original | 100% | 74.0% |
On-policy (40 responses sampled from the real MiMo-V2.6-Flash via API, only assistant tokens scored):
| group | tokens | original NLL | this model NLL | Δ | KL | top-1 agree |
|---|---|---|---|---|---|---|
| ALL | 24136 | 0.530 | 0.758 | +0.227 | 0.266 | 85.0% |
| code | 10627 | 0.511 | 0.742 | +0.231 | 0.261 | 85.7% |
| agent | 4786 | 0.695 | 0.933 | +0.237 | 0.302 | 82.1% |
| reasoning | 3815 | 0.313 | 0.410 | +0.097 | 0.125 | 91.3% |
| general | 4908 | 0.580 | 0.891 | +0.311 | 0.354 | 81.5% |
On-policy NLL is the most trustworthy number here: a compressed model that reproduces the original's own outputs has not drifted. Any perturbation of this MoE (even 8-bit attention) sits at KL≈0.4 on foreign text because top-8 routing flips, so only on-policy deltas are comparable across variants.
These are distribution-level numbers against the original model on the calibration domain; they are not benchmark scores. Expect a real capability loss versus the 4-bit original — measure on your task.
pip install -U mlx-lm
# default GPU wired limit is ~75% of RAM; allow the model + KV cache
sudo sysctl iogpu.wired_limit_mb=118000
hf download tacodevs/MiMo-V2.6-Flash-RL-MLX-REAP50-mxfp4-MTP --local-dir MiMo-V2.6-Flash-RL-MLX-REAP50-mxfp4-MTP
python -m mlx_lm generate --model MiMo-V2.6-Flash-RL-MLX-REAP50-mxfp4-MTP --prompt "Write a Python function that checks whether an integer is prime." --max-tokens 256 --temp 0.6
Close other memory-hungry apps first. The MTP / DFlash payloads are packaged for MiMo-aware runtimes; stock mlx-lm decodes serially and does not use them yet.
model-*.safetensors, model.safetensors.index.json, config.json — text model (this work)compression_alloc.json, compression_eval.json — per-layer precision map, pruned expert ids, evaluationmtp/, dflash/, omnimodal/, audio_tokenizer/ — upstream auxiliary weights (Vontra)Pipeline source: https://github.com/irvollo/mimo-mlx-compress (REAP saliency, sensitivity, MILP allocation, batched GPTQ, MLX packing).
Xiaomi MiMo team (model, MIT license); Vontra (MLX layout, MTP packaging); Cerebras (REAP); Deng et al. (GEMQ); Frantar et al. (GPTQ).
model-00008-of-00019.safetensors 4.9 GBmodel-00010-of-00019.safetensors 4.9 GBmodel-00013-of-00019.safetensors 4.9 GBmodel-00017-of-00019.safetensors 4.9 GBmodel-00004-of-00019.safetensors 4.9 GBmodel-00005-of-00019.safetensors 4.9 GBmodel-00011-of-00019.safetensors 4.9 GBmodel-00014-of-00019.safetensors 4.9 GBmodel-00007-of-00019.safetensors 4.9 GBmodel-00016-of-00019.safetensors 4.9 GBmodel-00002-of-00019.safetensors 4.9 GBmodel-00006-of-00019.safetensors 4.8 GBmodel-00012-of-00019.safetensors 4.8 GBmodel-00015-of-00019.safetensors 4.8 GBmodel-00003-of-00019.safetensors 4.8 GBmodel-00009-of-00019.safetensors 4.8 GBmodel-00018-of-00019.safetensors 4.8 GBmodel-00001-of-00019.safetensors 4.7 GBdflash/model.safetensors 2.9 GBaudio_tokenizer/model.safetensors 1.9 GBomnimodal/vision_encoder.safetensors 1.5 GBmodel-00019-of-00019.safetensors 570 MBmtp/model_mtp.safetensors 556 MBomnimodal/audio_encoder.safetensors 522 MBdflash/mask_embedding.pt 9.9 kB有文件?提交种子。
提交种子