MiMo-V2.5-Pro-MXFP4

Model Overview

• Model Architecture: MiMo-V2.5-Pro (MoE with multi-token-prediction head)
  • Input: Text
  • Output: Text
• Supported Hardware Microarchitecture: AMD MI3xx (gfx950)
• ROCm: 7.2
• Operating System(s): Linux
• Inference Engine: vLLM
• Model Optimizer: AMD-Quark
  • Quantized layers: MoE experts (*.experts.*.{gate,up,down}_proj) and dense MLPs (layers.0.mlp.*)
  • Weight quantization: OCP MXFP4, Static
  • Activation quantization: OCP MXFP4, Dynamic
• Calibration Dataset: None (MXFP4 weight quant uses per-block scaling; activations are dynamic)

This model was built from MiMo-V2.5-Pro by applying AMD-Quark for MXFP4 quantization. All attention layers (*self_attn*), the MoE router (*mlp.gate), the LM head, embeddings, norms, and the MTP layers (*mtp.layers*) are excluded from MXFP4.

Model Quantization

The model was quantized directly from the source FP8 release at XiaomiMiMo/MiMo-V2.5-Pro using AMD-Quark in --file2file_quantization mode (per-shard streaming, no full-model materialization). Both weights and activations of the targeted layers are quantized to MXFP4.

Quantization script

cd Quark/examples/torch/language_modeling/llm_ptq/
exclude_layers="*self_attn* *mlp.gate *mlp.gate.linear *lm_head *embed_tokens* *norm* *mtp.layers*"

python quantize_quark.py \
    --model_dir /path/to/MiMo-V2.5-Pro \
    --quant_scheme mxfp4 \
    --exclude_layers $exclude_layers \
    --output_dir amd/MiMo-V2.5-Pro-MXFP4 \
    --model_export hf_format \
    --file2file_quantization

Deployment

Use with vLLM

This model can be deployed efficiently using the vLLM backend.

Evaluation

The model was evaluated on GSM8K.

Accuracy

Reproduction

The results above were obtained using the lm-evaluation-harness framework, with vLLM and amd-quark compiled and installed from source.

lm_eval \
  --model vllm \
  --model_args pretrained=amd/MiMo-V2.5-Pro-MXFP4,trust_remote_code=True,tensor_parallel_size=8,dtype=auto,gpu_memory_utilization=0.9,max_model_len=4096,enforce_eager=False \
  --tasks gsm8k \
  --num_fewshot 5 \
  --batch_size auto

License

This model is a quantized derivative of XiaomiMiMo/MiMo-V2.5-Pro and is released under the same MIT license as the upstream model.

Modifications Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved.