Kimi-K3.1-Pro-pruned75

A calibration-free, expert-pruned checkpoint derived from moonshotai/Kimi-K3.1-Pro that removes 75% of the routed experts in every MoE layer. It is intended for vLLM serving and evaluation smoke tests on fewer GPUs, not for quality-sensitive use. Expect a large quality regression relative to the source model.

Pruning

• Routed experts per MoE layer: 896 → 224 (75% removed), across 92 MoE layers
• Ranking: signed sum of each expert's router row, sum(router.weight[i]), computed independently per layer
• Removed: pruned expert tensors, plus the matching rows of gate.weight and entries of gate.e_score_correction_bias. Expert indices are renumbered and text_config.num_experts is set to 224
• Unchanged: attention, shared experts, dense layer, vision tower, and the original MXFP4 quantization of the routed experts
• Calibration data: none
• Size: ~443 GB (vs. ~1.5 TB for the source)

Produced with the MagnitudeExpertPruner converter in compressed-tensors, with its default L1 scoring (weight.abs().sum(-1)) replaced by a signed sum. On this model the signed ranking scored much higher (GSM8K 43.7% vs. 14.4% for L1).

import torch
from compressed_tensors.entrypoints.convert import (
    MagnitudeExpertPruner,
    convert_checkpoint,
)
from compressed_tensors.entrypoints.convert.converters import (
    magnitude_expert_pruner as mep,
)
from safetensors import safe_open


def signed_sum_retained(router_names, weight_map, model_files, sparsity):
    retained = {}
    for name in router_names:
        with safe_open(model_files[weight_map[name]], framework="pt") as f:
            scores = f.get_tensor(name).float().sum(dim=-1)
        k = max(1, scores.shape[0] - round(sparsity * scores.shape[0]))
        retained[name] = sorted(torch.topk(scores, k).indices.tolist())
    return retained


mep._compute_retained_experts = signed_sum_retained

MODEL_ID = "moonshotai/Kimi-K3.1-Pro"
convert_checkpoint(
    model_stub=MODEL_ID,
    save_directory="Kimi-K3.1-Pro-pruned75",
    converter=MagnitudeExpertPruner.from_pretrained(
        MODEL_ID,
        router_pattern=r"language_model\..*\.block_sparse_moe\.gate\.weight$",
        expert_pattern=r"language_model\..*\.block_sparse_moe\.experts\.\d+\.w[123]\.",
        aux_pattern=r"block_sparse_moe\.gate\.e_score_correction_bias$",
        sparsity=0.75,
        num_experts_config_key="num_experts",
    ),
    device="cpu",
)

Evaluation

GSM8K 5-shot via lm_eval (local-completions) against vLLM, TP=4 on NVIDIA B300:

Usage

vllm serve vllm-project/Kimi-K3.1-Pro-pruned75 \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3 \
  --reasoning-parser kimi_k3

License

Same as the source model: see the Kimi K3.1 Pro license.