MiMo-V2.6-Flash-RL, EXL3 3.0 bpw

XiaomiMiMo/MiMo-V2.6-Flash-RL (309B MoE, 15B active) quantised to EXL3 at 3.0 bits per weight, made to run across two NVIDIA DGX Sparks (GB10) with tensor parallelism. The recipe that serves it, with every measurement below as raw JSON: ajensenwaud/MiMo-V2.6-2x-DGX-Spark-TensorFold.

Text only: the vision tower and audio heads are not in this pack.
Weights 113 GB, 17 shards (60 GiB a Spark at TP=2)
Bitrate 3.03 bpw (excluding head); head 8 bpw; MTP heads 4 bpw
Conversion exllamav3 v1.6.0, -b 3.0 -hb 8 -mb 4 -hq -cr 250 -cc 2048 -cb mul1, source @ 5711b268
Drafter dflash/: Xiaomi's DFlash drafter, 4 bpw EXL3, as published in benthecarman/MiMo-V2.6-Flash-RL-exl3 @ 5ae87b83

Quality

Served on two Sparks with the recipe's engine (TensorFold mimo_v2), the same 500 items a task for every pack, thinking off, greedy. The 2.27 bpw pack is benthecarman's, the 4.0 bpw pack ours with the same settings as this one.

Pack GSM8K MMLU-Pro Wikitext-2 perplexity (64 x 2048)
2.27 bpw 95.4% 70.0% 5.402
3.0 bpw (this) 96.0% 73.4% 5.069
4.0 bpw 96.6% 73.6% 4.979

Paired on identical items, MMLU-Pro: 3.0 vs 2.27 +3.4 points (McNemar p = 0.04); 4.0 vs 3.0 +0.2 (p = 0.89). GSM8K differences are within noise. Not measured: code tasks, GPQA, anything with thinking on.

Speed (two DGX Sparks, the recipe, one stream, 512-token replies)

Task Plain decode With the DFlash drafter
coding 45.4 tok/s 77.6 tok/s
code edit 44.7 80.8
reasoning 45.3 72.8
prose 45.5 46.0

Drafted replies are byte-identical to plain ones. Prefill about 1,200 tok/s at 30-60K-token prompts.

Use

With the recipe (two Sparks): PACK=3.0 ./start.sh after scripts/fetch_pack.sh. The pack is a standard exllamav3 v1.6.0 conversion of the mimo_v2 architecture; it was evaluated only with the recipe's engine, not with exllamav3 itself (at 113 GB it does not fit one GB10 with room to run).

Acknowledgements

Xiaomi MiMo for the model and the DFlash drafter; turboderp for exllamav3, which made these packs; benthecarman for the 2.27 bpw pack, the conversion settings reused here and the repaired drafter; Ash Hart for TensorFold, the engine the packs were evaluated on; Mia's AI Lab for the two-Spark recipe format. Evaluation data: GSM8K (Cobbe et al., MIT), MMLU-Pro (TIGER-Lab, MIT), WikiText-2 (Merity et al., CC BY-SA 3.0). Full list in the recipe's CREDITS.md.

Licence

MIT, as the source model. The dflash/ drafter is Xiaomi's, taken unchanged from benthecarman's MIT-licensed pack.