Mistral Large 4 REAP 667B-A52B (NVFP4)

An expert-pruned version of Mistral Large 4 NVFP4. Each of the 71 MoE layers keeps 80 of its 128 routed experts (37.5% pruned). Total parameters drop from 1,053B to 667B and the checkpoint from 610 GB to 393 GB. The router still picks 4 experts per token, so active parameters stay at about 52B.

We made it to run on four NVIDIA DGX Spark boxes (4 × 128 GB), which can't hold the full NVFP4 release.

How the experts were chosen

We scored experts with REAP (Lasby et al.): for each expert, the mean over the tokens routed to it of router weight × the L2 norm of the expert's output. In each layer the 80 highest-scoring experts stay.

The scores come from 1.41M tokens run through the full, unpruned NVFP4 model. Every document was rendered with the model's own chat template, with reasoning traces in [THINK] blocks:

On that data the kept experts carry 84.5% of the router-weighted expert output on average (worst layer 76.4%) and receive 66.5% of the routing decisions. One expert in the whole model was never routed to.

Pruning is weight surgery only. Kept experts are renumbered in their original order, and each router (layers.N.gate.weight) keeps the matching rows. The NVFP4 weights and scales are copied byte for byte with no requantization or fine-tuning. Attention, the shared experts, the vision encoder and the DSpark draft head are unchanged.

prune_keep.json lists the original expert ids kept in each layer. reap/ has the summed calibration scores and the scripts.

Evaluation

We haven't run benchmarks yet. So far the model is only spot-checked: it writes correct Python, solves short arithmetic word problems, makes well-formed tool calls through vLLM's mistral tool parser, and its reasoning reads coherently. Expect some quality loss against the full model, most likely on less common languages and domains the calibration set covers thinly.

Serving with vLLM

Use Mistral's image, vllm/vllm-openai:mistral-large-4. On four DGX Sparks we run tensor parallel 2 within each cabled pair and pipeline parallel 2 across the pairs:

docker run -d --gpus all --net=host --ipc=host --shm-size 32g \
  -v /path/to/model:/model:ro \
  -e VLLM_PP_LAYER_PARTITION=35,37 \
  vllm/vllm-openai:mistral-large-4 \
  --model /model --served-model-name mistral-large-4-reap \
  --tensor-parallel-size 2 --pipeline-parallel-size 2 \
  --nnodes 4 --node-rank $RANK --master-addr $HEAD_IP \
  --kv-cache-dtype fp8_ds_mla \
  --tool-call-parser mistral --reasoning-parser mistral --enable-auto-tool-choice \
  --max-model-len 65536 --gpu-memory-utilization 0.86

Add --headless on ranks 1 to 3. Layers 39, 43, ... 71 reuse the KV cache of layer 35, so a pipeline stage boundary must sit at or before layer 35. VLLM_PP_LAYER_PARTITION=35,37 does that. With that split the second stage holds about 95.6 GiB of weights per Spark, which leaves about 297k tokens of KV cache at 0.86 memory utilization.

On our four Sparks, where the two pairs only reach each other over Wi-Fi and NCCL runs over TCP, we measured 5.7 tokens/s for one request, 14.7 tokens/s total at 8 concurrent requests and 20.3 at 16.

License

Mistral Ecosystem Open-Weight License, as in the base model. The license text is in LICENSE. Attribution notice: 2026 - Mistral.