Xiaomi MiMo-V2.6-Flash REAP-50 — GGUF

Official GGUF quantisations of MiMo-V2.6-Flash-REAP50, a 50% routed-expert pruned checkpoint of XiaomiMiMo/MiMo-V2.6-Flash created with REAP and HOPE second-order saliency pruning.

• Base HF Checkpoint: patrickbdevaney/MiMo-V2.6-Flash-REAP50
• Experts Retained: 128 of 256 routed experts per layer across 47 MoE layers (1 dense layer, 47 MoE layers).
• Base Architecture: Native packed MXFP4 (U8, block size 32) experts with unquantized pure BF16 attention and embeddings.
• Towers Included: Vision & Audio multimodal projectors (mmproj) and Multi-Token Prediction speculative draft heads (mtp).

---

Quantization Ladder

Supporting Towers (Vision, Audio & MTP)

---

Key Features

1. Native MXFP4 MoE Preservation: In the base model, 92.9% of weights are stored as native packed mxfp4 (32 block size). Our GGUF converter natively repacks these blocks directly into GGMLQuantizationType.MXFP4, avoiding costly lossy dequantization cycles while preserving exact native numerical precision.

2. Multimodal Projectors (mmproj): Xiaomi MiMo-V2.6-Flash incorporates both visual and audio processing towers:

   • Vision encoder (28-layer ViT, 560px patch representation)
   • Audio tokenizer / RVQ speech representations Both are packed into standard GGUF multimodal projectors (mmproj-*-BF16.gguf and mmproj-*-Q8_0.gguf) compatible with llama.cpp's multimodal pipeline.

3. Multi-Token Prediction (mtp): MiMo-V2.6-Flash includes 3 trained MTP layers for speculative decoding. We ship standalone MTP draft models (mtp-*-BF16.gguf and mtp-*-Q8_0.gguf) that can be loaded alongside the trunk model with --draft-model to accelerate generation.

---

Running with llama.cpp

1. Standard Text Inference (Optimal Hybrid Q2_K)

./llama-cli \
    -m MiMo-V2.6-Flash-REAP50-Q2_K.gguf \
    -p "You are MiMo, an AI assistant developed by Xiaomi. Explain how MoE expert pruning works:" \
    -n 512 --temp 0.6

Or run the flagship bit-for-bit native MXFP4 checkpoint:

./llama-cli \
    -m MiMo-V2.6-Flash-REAP50-MXFP4_MOE.gguf \
    -p "You are MiMo, an AI assistant developed by Xiaomi. Explain how MoE expert pruning works:" \
    -n 512 --temp 0.6

2. Speculative Decoding with MTP Draft Head

./llama-cli \
    -m MiMo-V2.6-Flash-REAP50-Q2_K.gguf \
    --draft-model mtp-MiMo-V2.6-Flash-REAP50-Q8_0.gguf \
    -p "Explain quantum teleportation in detail:" \
    -n 512

3. Multimodal Inference (Vision & Audio)

./llama-cli \
    -m MiMo-V2.6-Flash-REAP50-Q2_K.gguf \
    --mmproj mmproj-MiMo-V2.6-Flash-REAP50-Q8_0.gguf \
    --image input.jpg \
    -p "Describe the contents of this image in detail."

4. OpenAI-Compatible API Server

./llama-server \
    -m MiMo-V2.6-Flash-REAP50-Q2_K.gguf \
    --mmproj mmproj-MiMo-V2.6-Flash-REAP50-Q8_0.gguf \
    --port 8080 \
    -ngl 99

---

Background & Pruning Method

Pruned using HOPE (Higher-Order Pruning of Experts) over a diverse calibration corpus spanning code, math, conversational text, and multimodal reasoning tasks. Rather than relying solely on first-order activation frequencies, HOPE accounts for inter-expert interaction terms: $$\Delta \mathcal{L} \approx \sum_{i} g_i^T \Delta w_i + \frac{1}{2} \sum_{i,j} \Delta w_i^T H_{ij} \Delta w_j$$ By computing cross-expert Hessian blocks during the calibration pass, 128 experts per layer were optimally selected to minimize perplexity loss under 50% parameter reduction.

---

Created by patrickbdevaney.