Kimi-K3.1-Pro-pruned75

This is a calibration-free, safetensors-only expert-pruned checkpoint derived from moonshotai/Kimi-K3.1-Pro. It removes 75% of the routed experts, following the same method as mgoin/Kimi-K3-pruned75. It is intended for vLLM serving and evaluation smoke tests. Accurate calibration-based pruning, such as REAP in llm-compressor, is a separate line of work.

This is an experimental prototype, not a REAP-pruned checkpoint. Expect a large quality regression relative to the source model.

Pruning

• Source experts per MoE layer: 896
• Retained experts per MoE layer: 224 (25% retained, 75% removed)
• Ranking: descending signed sum(router.weight[i]), independently per layer
• Removed data: pruned expert tensors and their corresponding router rows
• Router and expert indices were compacted, and num_experts was updated to 224
• Calibration data: none
• MXFP4 expert weights were copied with their scale/packing intact (not dequantized)
• On-disk size: about 443G across 96 safetensors shards plus the weight index

Only routed experts were changed. Dense first-layer MLP, shared experts, attention / DSA / indexer, latent up-down projections, vision weights, and the tokenizer were left untouched.

Validated vLLM run

The checkpoint loaded and served with the default vLLM loader on one node with eight NVIDIA H200 GPUs:

• Hardware: 8x NVIDIA H200
• Parallelism: TP=8 on one node
• Served model name: kimi-k3.1-pro-pruned75
• Steady GPU memory: about 127,797 MiB per GPU

vllm serve DaoCloud/Kimi-K3.1-Pro-pruned75 \
  --served-model-name kimi-k3.1-pro-pruned75 \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --host 0.0.0.0 \
  --port 8000

GET /v1/models returned HTTP 200 after load.

GSM8K

Evaluation used a local OpenAI-compatible /v1/completions server and the full GSM8K test split, aligned with the mgoin harness settings:

• 1,319 questions, 5-shot, max_tokens=256, temperature=0, seed=42

For reference, mgoin/Kimi-K3-pruned75 scored 34.4200% (454/1319) with the same shot/token/temperature/seed settings on the original Kimi-K3 pruned75 checkpoint (different base model and hardware).

License

This derivative is distributed under the upstream Kimi K3.1 Pro License.