MiMo-V2.6-Flash-Sushi-2.3bpw
XiaomiMiMo/MiMo-V2.6-Flash-MOPD packed for sushi, a native inference engine for Apple Silicon. Target: a 128 GB Mac. The weights alone take about 90 GB of GPU memory, so it does not fit on a 96 GB Mac.
| part | how it is stored |
|---|---|
| routed experts (47 MoE layers x 256, top-8) | EXL3 at 2.25 bits per weight (the last layer at 4), made with our own converter |
| dense part (attention, layer-0 MLP) | FP8 as released, with its 128x128 block scales; output projections, embedding and lm_head 8-bit affine |
| vision tower (MiMo-ViT) | included: image input |
| MTP heads | included (3 layers) |
| context | up to 1,048,576 tokens |
| disk | ~96 GiB |
sushi only. This pack needs sushi v1.1.0 or later on Apple Silicon with macOS 26.2 or later. It does not load in transformers, vLLM, mlx-lm or exllamav3. Audio and video input are not supported.
How to run it (install, launch flags, memory and context per Mac): sushi usage.
dflash/ (the DFlash drafter) and audio_tokenizer/ are Xiaomi's originals, unchanged, for runtimes that use them;
sushi does not load either.
Quality
KLD against the original MOPD checkpoint (its FP8 dense part as stored, no quantized KV of its own): 16 prompts x 512 tokens, scored to the first EOS (8067 positions), KV cache at kv8. Lower is better.
| pack | weights in GPU memory | KLD | top-1 agreement |
|---|---|---|---|
| MiMo-V2.6-Flash-Sushi-2.3bpw | ~90 GB | 0.0860 | 91.95% |
Hard questions
LLMProbe's Hard 50 set (MMLU-Pro, OlympiadBench, LiveBench, NIST Juliet): 42 correct, 7 incorrect, 1 partial (hit the token cap), 84.0%. llmprobe 0.6.12, xhigh effort, 12k reasoning / 16k output tokens, 8-bit KV and MTP.
Credits and license
- Xiaomi MiMo team: the base model, XiaomiMiMo/MiMo-V2.6-Flash-MOPD, released under the MIT License; this pack is a derivative under the same license.
- turboderp: the EXL3 format.
- ddalcu: mlx-serve. sushi is a fork of it.