MiMo-V2.6-Flash-REAP50
XiaomiMiMo/MiMo-V2.6-Flash-RL with 50% of its routed experts removed — 256
experts per layer down to 128 — so that it fits and serves on a single
NVIDIA Jetson AGX Thor (117 GiB unified memory). 86.1 GiB across 65 shards.
Vision, audio and video input are preserved; audio_tokenizer/ ships with the checkpoint.
How the experts were chosen
Not by activation frequency. Expert saliency was accumulated over a calibration corpus and the prune set was solved with HOPE, which minimises the output error a prune set actually causes including the interaction terms between experts — REAP is the same objective with the off-diagonal zeroed, and that off-diagonal cannot be recovered after the pass.
| setting | value |
|---|---|
| objective | hope |
| saliency criterion | reap_1_1_1 |
| prune ratio | 0.50, uniform across layers |
| per-domain protection | top 8% of every domain held out of the prune set |
| worst domain retained | 0.9935 (audio) |
| mean retained | 0.9964 |
| HOPE objective pᵀFp | 0.01624 |
Selection is scored per domain and ranked by the worst one, never the mean: an average is how a criterion that destroys one capability outscores one that preserves all of them.
Retained gated output mass, by calibration domain
| domain | retention |
|---|---|
| audio | 0.9935 |
| image | 0.9937 |
| video | 0.9938 |
| science | 0.9973 |
| math | 0.9974 |
| finance | 0.9974 |
| ballast | 0.9976 |
| code | 0.9983 |
| agentic | 0.9983 |
Routers
Pruning an expert leaves its router column behind. The routers were refitted by output matching against the unpruned teacher, routers only, every expert frozen: 47 routers refitted; all 47 kept at the teacher weights (the fit did not beat the baseline).
A refit that failed to beat the untouched baseline was discarded in favour of the baseline, so no router here is worse than simply slicing the teacher's.
Limitations
- Calibration was English/Chinese text, code, math, science, finance, agentic traces, and image/audio/video captions. Domains outside that mix were not measured.
- The
dflash/speculative-decoding draft head from the source repo is not included: it was trained against the unpruned expert set and is not valid for this checkpoint. - Pruned MoE experts do not come back. This is a lossy, irreversible transform of the base model.
Provenance
Produced by patrickbdevaney/xiaomi-2.6-flash-REAP on a single Jetson AGX Thor. MIT, inherited from the base model — attribution to Xiaomi MiMo.