XiaomiMiMo/MiMo-V2.6-Pro-MOPD in Jarrelscy's ARVQ / NVFP4 hybrid format, NVFP4 hot set widened from 5% to 21% of routed experts. Four DGX Sparks (GB10), TP4, 1M context.
This checkpoint is stock MOPD C3. It is not abliterated.
Recipe (image v4, full-speed defaults): keys-MiMo-V2.6-Pro-MOPD-C3-Jarrelscy-ARVQ-4-DGX-Sparks-1M-Context.
| Part | Source |
|---|---|
| Non-expert weights (attention, norms, embeddings, LM head, dense layer 0) | MOPD, exact. Attention o_proj in FP8 128×128 blocks |
| MTP draft heads | identical in RL and MOPD |
| 5,560 hot routed experts (21%), NVFP4 | MOPD, requantized with Jarrelscy's hybrid.quantize |
| 20,936 cold routed experts, ARVQ 2-bit | Jarrelscy's published PV-fitted RL encoding, unchanged |
Hot set: Jarrelscy's prepare_hybrid.select_hot, count 5,560 instead of 1,325, layers 27–68. See allocation.json and config.json (aqlm_layer_books).
Live TP4, idle endpoint. Speed: 512 new tokens, temperature 1.0. Refusal/cyber: greedy, 192 tokens.
| Prose | 24.2 tok/s (1.91 tok/pass) |
| Code | 32.2 tok/s (2.60 tok/pass) |
| 4 requests, aggregate | 46.3 tok/s |
| Prefill 9.5K / 38K | 883 / 1074 tok/s |
| Refusal 32 | 5/32 bypass (stock MOPD) |
| Cyber 22 | 6/22 bypass |
Tool-call repetition (435 Hermes turns, server defaults): 7.4% duplicate-call turns, 0.9% flood (32+ calls). GSM8K 96.8 / HumanEval 93.9 / MMLU-Pro 77.4.
Weights ~83.2 GiB/rank. Measured KV pool 1,282,005 tokens (still covers 1M context).
hf download drowzeys/keys-MiMo-V2.6-Pro-MOPD-Jarrelscy-ARVQ-hybrid21 --local-dir /path/to/mimo-mopd
git clone https://github.com/drowzeys/keys-MiMo-V2.6-Pro-MOPD-C3-Jarrelscy-ARVQ-4-DGX-Sparks-1M-Context
export MASTER_ADDR=<rank-0 IP>
# ranks 1-3 first, then rank 0. GPU memory fraction is fixed at 0.85.
bash serve/launch-rank.sh <node-ip> <1|2|3> <RoCE-GID-index> /path/to/mimo-mopd headless
bash serve/launch-rank.sh <node-ip> 0 <RoCE-GID-index> /path/to/mimo-mopd api
Defaults: MTP k=2 (all three heads, non-chain), torch.compile + CUDA graphs, batched ARVQ prefill, 1M context, 4 seqs, both CX-7 RoCE paths.
Jarrelscy format, codebooks, PV fits, and vLLM fork. Xiaomi MOPD/RL weights (MIT).
backbone-001.safetensors 23 GBdflash/dflash_draft_model.safetensors 5.5 GBroster-layer-048.safetensors 4.1 GBroster-layer-049.safetensors 4.1 GBroster-layer-050.safetensors 4.1 GBroster-layer-051.safetensors 4.1 GBroster-layer-052.safetensors 4.1 GBroster-layer-053.safetensors 4.1 GBroster-layer-054.safetensors 4.1 GBroster-layer-055.safetensors 4.1 GBroster-layer-056.safetensors 4.1 GBroster-layer-057.safetensors 4.1 GBroster-layer-058.safetensors 4.1 GBroster-layer-059.safetensors 4.1 GBroster-layer-060.safetensors 4.1 GBroster-layer-061.safetensors 4.1 GBroster-layer-062.safetensors 4.1 GBroster-layer-063.safetensors 4.1 GBroster-layer-064.safetensors 4.1 GBroster-layer-065.safetensors 4.1 GBroster-layer-066.safetensors 4.1 GBroster-layer-067.safetensors 4.1 GBroster-layer-068.safetensors 4.1 GBroster-layer-069.safetensors 4.1 GBroster-layer-047.safetensors 3.9 GBroster-layer-045.safetensors 3.6 GBroster-layer-046.safetensors 3.6 GBroster-layer-044.safetensors 3.3 GBroster-layer-042.safetensors 2.7 GBarvq-layer-010-gateup.safetensors 2.6 GBarvq-layer-011-gateup.safetensors 2.6 GBarvq-layer-012-gateup.safetensors 2.6 GBarvq-layer-013-gateup.safetensors 2.6 GBarvq-layer-014-gateup.safetensors 2.6 GBarvq-layer-015-gateup.safetensors 2.6 GBarvq-layer-016-gateup.safetensors 2.6 GBarvq-layer-017-gateup.safetensors 2.6 GBarvq-layer-018-gateup.safetensors 2.6 GBarvq-layer-019-gateup.safetensors 2.6 GBarvq-layer-020-gateup.safetensors 2.6 GBarvq-layer-002-gateup.safetensors 2.6 GBarvq-layer-003-gateup.safetensors 2.6 GBarvq-layer-004-gateup.safetensors 2.6 GBarvq-layer-005-gateup.safetensors 2.6 GBarvq-layer-006-gateup.safetensors 2.6 GBarvq-layer-007-gateup.safetensors 2.6 GBarvq-layer-008-gateup.safetensors 2.6 GBarvq-layer-009-gateup.safetensors 2.6 GBarvq-layer-021-gateup.safetensors 2.6 GBarvq-layer-023-gateup.safetensors 2.6 GBarvq-layer-024-gateup.safetensors 2.6 GBarvq-layer-025-gateup.safetensors 2.6 GBarvq-layer-026-gateup.safetensors 2.6 GBarvq-layer-001-gateup.safetensors 2.6 GBarvq-layer-022-gateup.safetensors 2.6 GBarvq-layer-027-gateup.safetensors 2.6 GBarvq-layer-030-gateup.safetensors 2.6 GBarvq-layer-028-gateup.safetensors 2.5 GBarvq-layer-029-gateup.safetensors 2.5 GBarvq-layer-031-gateup.safetensors 2.5 GBarvq-layer-033-gateup.safetensors 2.5 GBarvq-layer-032-gateup.safetensors 2.5 GBroster-layer-043.safetensors 2.5 GBarvq-layer-034-gateup.safetensors 2.5 GBarvq-layer-035-gateup.safetensors 2.5 GBbackbone-002.safetensors 2.5 GBarvq-layer-036-gateup.safetensors 2.4 GBroster-layer-041.safetensors 2.4 GBarvq-layer-037-gateup.safetensors 2.4 GBarvq-layer-038-gateup.safetensors 2.3 GBarvq-layer-039-gateup.safetensors 2.1 GBbackbone-000.safetensors 2.0 GBarvq-layer-040-gateup.safetensors 2.0 GBaudio_tokenizer/model.safetensors 1.9 GBarvq-layer-041-gateup.safetensors 1.8 GBroster-layer-040.safetensors 1.8 GBarvq-layer-043-gateup.safetensors 1.8 GBarvq-layer-042-gateup.safetensors 1.7 GBarvq-layer-044-gateup.safetensors 1.5 GBroster-layer-039.safetensors 1.5 GBarvq-layer-046-gateup.safetensors 1.4 GBarvq-layer-045-gateup.safetensors 1.4 GBarvq-layer-047-gateup.safetensors 1.3 GBarvq-layer-002-down.safetensors 1.3 GBarvq-layer-003-down.safetensors 1.3 GBarvq-layer-004-down.safetensors 1.3 GBarvq-layer-005-down.safetensors 1.3 GBarvq-layer-006-down.safetensors 1.3 GBarvq-layer-007-down.safetensors 1.3 GBarvq-layer-008-down.safetensors 1.3 GBarvq-layer-009-down.safetensors 1.3 GBarvq-layer-010-down.safetensors 1.3 GBarvq-layer-011-down.safetensors 1.3 GBarvq-layer-012-down.safetensors 1.3 GBarvq-layer-013-down.safetensors 1.3 GBarvq-layer-014-down.safetensors 1.3 GBarvq-layer-015-down.safetensors 1.3 GBarvq-layer-016-down.safetensors 1.3 GBarvq-layer-017-down.safetensors 1.3 GBarvq-layer-018-down.safetensors 1.3 GBarvq-layer-019-down.safetensors 1.3 GBarvq-layer-020-down.safetensors 1.3 GBarvq-layer-001-down.safetensors 1.3 GBarvq-layer-021-down.safetensors 1.3 GBarvq-layer-023-down.safetensors 1.3 GBarvq-layer-024-down.safetensors 1.3 GBarvq-layer-025-down.safetensors 1.3 GBarvq-layer-026-down.safetensors 1.3 GBarvq-layer-048-gateup.safetensors 1.3 GBarvq-layer-049-gateup.safetensors 1.3 GBarvq-layer-050-gateup.safetensors 1.3 GBarvq-layer-051-gateup.safetensors 1.3 GBarvq-layer-052-gateup.safetensors 1.3 GBarvq-layer-053-gateup.safetensors 1.3 GBarvq-layer-054-gateup.safetensors 1.3 GBarvq-layer-055-gateup.safetensors 1.3 GBarvq-layer-056-gateup.safetensors 1.3 GBarvq-layer-057-gateup.safetensors 1.3 GBarvq-layer-058-gateup.safetensors 1.3 GBarvq-layer-059-gateup.safetensors 1.3 GBarvq-layer-060-gateup.safetensors 1.3 GBarvq-layer-061-gateup.safetensors 1.3 GBarvq-layer-062-gateup.safetensors 1.3 GBarvq-layer-063-gateup.safetensors 1.3 GBarvq-layer-064-gateup.safetensors 1.3 GBarvq-layer-065-gateup.safetensors 1.3 GBarvq-layer-066-gateup.safetensors 1.3 GBarvq-layer-067-gateup.safetensors 1.3 GBarvq-layer-068-gateup.safetensors 1.3 GBarvq-layer-069-gateup.safetensors 1.3 GBarvq-layer-022-down.safetensors 1.3 GBarvq-layer-027-down.safetensors 1.3 GBarvq-layer-030-down.safetensors 1.3 GBarvq-layer-028-down.safetensors 1.3 GBarvq-layer-029-down.safetensors 1.3 GBarvq-layer-031-down.safetensors 1.3 GBarvq-layer-033-down.safetensors 1.3 GBarvq-layer-032-down.safetensors 1.3 GBarvq-layer-034-down.safetensors 1.2 GBarvq-layer-035-down.safetensors 1.2 GBarvq-layer-036-down.safetensors 1.2 GBarvq-layer-037-down.safetensors 1.2 GBarvq-layer-038-down.safetensors 1.2 GBarvq-layer-039-down.safetensors 1.0 GBarvq-layer-040-down.safetensors 1000 MBarvq-layer-041-down.safetensors 903 MBarvq-layer-043-down.safetensors 893 MBarvq-layer-042-down.safetensors 863 MBroster-layer-038.safetensors 807 MBarvq-layer-044-down.safetensors 769 MBarvq-layer-046-down.safetensors 716 MBarvq-layer-045-down.safetensors 712 MBarvq-layer-047-down.safetensors 662 MBarvq-layer-048-down.safetensors 642 MBarvq-layer-049-down.safetensors 642 MBarvq-layer-050-down.safetensors 642 MBarvq-layer-051-down.safetensors 642 MBarvq-layer-052-down.safetensors 642 MBarvq-layer-053-down.safetensors 642 MBarvq-layer-054-down.safetensors 642 MBarvq-layer-055-down.safetensors 642 MBarvq-layer-056-down.safetensors 642 MBarvq-layer-057-down.safetensors 642 MBarvq-layer-058-down.safetensors 642 MBarvq-layer-059-down.safetensors 642 MBarvq-layer-060-down.safetensors 642 MBarvq-layer-061-down.safetensors 642 MBarvq-layer-062-down.safetensors 642 MBarvq-layer-063-down.safetensors 642 MBarvq-layer-064-down.safetensors 642 MBarvq-layer-065-down.safetensors 642 MBarvq-layer-066-down.safetensors 642 MBarvq-layer-067-down.safetensors 642 MBarvq-layer-068-down.safetensors 642 MBarvq-layer-069-down.safetensors 642 MBroster-layer-037.safetensors 616 MBroster-layer-036.safetensors 446 MBroster-layer-035.safetensors 319 MBroster-layer-034.safetensors 276 MBroster-layer-032.safetensors 149 MBroster-layer-031.safetensors 85 MBroster-layer-033.safetensors 85 MBroster-layer-028.safetensors 64 MBroster-layer-029.safetensors 64 MBroster-layer-027.safetensors 42 MBroster-layer-030.safetensors 42 MBroster-layer-022.safetensors 21 MBdflash/mask_embedding.pt 14 kBroster-layer-010.safetensors 1.1 kBroster-layer-011.safetensors 1.1 kBroster-layer-012.safetensors 1.1 kBroster-layer-013.safetensors 1.1 kBroster-layer-014.safetensors 1.1 kBroster-layer-015.safetensors 1.1 kBroster-layer-016.safetensors 1.1 kBroster-layer-017.safetensors 1.1 kBroster-layer-018.safetensors 1.1 kBroster-layer-019.safetensors 1.1 kBroster-layer-020.safetensors 1.1 kBroster-layer-021.safetensors 1.1 kBroster-layer-023.safetensors 1.1 kBroster-layer-024.safetensors 1.1 kBroster-layer-025.safetensors 1.1 kBroster-layer-026.safetensors 1.1 kBroster-layer-001.safetensors 1.1 kBroster-layer-002.safetensors 1.1 kBroster-layer-003.safetensors 1.1 kBroster-layer-004.safetensors 1.1 kBroster-layer-005.safetensors 1.1 kBroster-layer-006.safetensors 1.1 kBroster-layer-007.safetensors 1.1 kBroster-layer-008.safetensors 1.1 kBroster-layer-009.safetensors 1.1 kB有文件?提交种子。
提交种子