𓌳 REAP𓌳 the Experts: Why Pruning Prevails for One-Shot MoE Compression
Introducing GLM-4.6-REAP-268B-A32B, a memory-efficient compressed variant of GLM-4.6 that maintains near-identical performance while being 25% lighter.
Note: this is a BF16 version for more accurate downstream low-bit quantization. An FP8 version is also available on HF.
This model was created using REAP (Router-weighted Expert Activation Pruning), a novel expert pruning method that selectively removes redundant experts while preserving the router's independent control over remaining experts. Key features include:
GLM-4.6-REAP-268B-A32B has the following specifications:
TBD for BF16 model. Evalulation results available for the FP8 variant.
For more details on the evaluation setup, refer to the REAP arXiv preprint.
You can deploy the model directly using the latest vLLM (v0.11.0), no source modifications or custom patches required.
vllm serve cerebras/GLM-4.6-REAP-268B-A32B \
--tensor-parallel-size 8 \
--tool-call-parser glm45 \
--enable-auto-tool-choice \
--enable-expert-parallel
If you encounter insufficient memory when running this model, you might need to set a lower value for --max-num-seqs flag (e.g. set to 64).
This checkpoint was created by applying the REAP (Router-weighted Expert Activation Pruning) method uniformly across all Mixture-of-Experts (MoE) blocks of GLM-4.6, with a 25% pruning rate.
REAP selects experts to prune based on a novel saliency criterion that considers both:
This dual consideration ensures that experts contributing minimally to the layer's output are pruned, while preserving those that play critical roles in the model's computations.
The model was calibrated using a diverse mixture of domain-specific datasets including:
📚 For more details, refer to the following resources:
This model is derived from
zai-org/GLM-4.6
and distributed under the MIT license.
If you use this checkpoint, please cite the REAP paper:
@article{lasby-reap,
title={REAP the Experts: Why Pruning Prevails for One-Shot MoE compression},
author={Lasby, Mike and Lazarevich, Ivan and Sinnadurai, Nish and Lie, Sean and Ioannou, Yani and Thangarasa, Vithursan},
journal={arXiv preprint arXiv:2510.13999},
year={2025}
}
model-00014-of-00108.safetensors 5.0 GBmodel-00020-of-00108.safetensors 5.0 GBmodel-00026-of-00108.safetensors 5.0 GBmodel-00032-of-00108.safetensors 5.0 GBmodel-00038-of-00108.safetensors 5.0 GBmodel-00011-of-00108.safetensors 5.0 GBmodel-00012-of-00108.safetensors 5.0 GBmodel-00013-of-00108.safetensors 5.0 GBmodel-00017-of-00108.safetensors 5.0 GBmodel-00018-of-00108.safetensors 5.0 GBmodel-00019-of-00108.safetensors 5.0 GBmodel-00023-of-00108.safetensors 5.0 GBmodel-00024-of-00108.safetensors 5.0 GBmodel-00025-of-00108.safetensors 5.0 GBmodel-00029-of-00108.safetensors 5.0 GBmodel-00030-of-00108.safetensors 5.0 GBmodel-00031-of-00108.safetensors 5.0 GBmodel-00035-of-00108.safetensors 5.0 GBmodel-00036-of-00108.safetensors 5.0 GBmodel-00037-of-00108.safetensors 5.0 GBmodel-00041-of-00108.safetensors 5.0 GBmodel-00042-of-00108.safetensors 5.0 GBmodel-00043-of-00108.safetensors 5.0 GBmodel-00044-of-00108.safetensors 5.0 GBmodel-00047-of-00108.safetensors 5.0 GBmodel-00048-of-00108.safetensors 5.0 GBmodel-00049-of-00108.safetensors 5.0 GBmodel-00050-of-00108.safetensors 5.0 GBmodel-00053-of-00108.safetensors 5.0 GBmodel-00054-of-00108.safetensors 5.0 GBmodel-00055-of-00108.safetensors 5.0 GBmodel-00056-of-00108.safetensors 5.0 GBmodel-00059-of-00108.safetensors 5.0 GBmodel-00060-of-00108.safetensors 5.0 GBmodel-00061-of-00108.safetensors 5.0 GBmodel-00062-of-00108.safetensors 5.0 GBmodel-00065-of-00108.safetensors 5.0 GBmodel-00066-of-00108.safetensors 5.0 GBmodel-00067-of-00108.safetensors 5.0 GBmodel-00068-of-00108.safetensors 5.0 GBmodel-00071-of-00108.safetensors 5.0 GBmodel-00072-of-00108.safetensors 5.0 GBmodel-00073-of-00108.safetensors 5.0 GBmodel-00074-of-00108.safetensors 5.0 GBmodel-00077-of-00108.safetensors 5.0 GBmodel-00078-of-00108.safetensors 5.0 GBmodel-00079-of-00108.safetensors 5.0 GBmodel-00080-of-00108.safetensors 5.0 GBmodel-00083-of-00108.safetensors 5.0 GBmodel-00084-of-00108.safetensors 5.0 GBmodel-00085-of-00108.safetensors 5.0 GBmodel-00086-of-00108.safetensors 5.0 GBmodel-00089-of-00108.safetensors 5.0 GBmodel-00090-of-00108.safetensors 5.0 GBmodel-00091-of-00108.safetensors 5.0 GBmodel-00092-of-00108.safetensors 5.0 GBmodel-00095-of-00108.safetensors 5.0 GBmodel-00096-of-00108.safetensors 5.0 GBmodel-00097-of-00108.safetensors 5.0 GBmodel-00098-of-00108.safetensors 5.0 GBmodel-00101-of-00108.safetensors 5.0 GBmodel-00102-of-00108.safetensors 5.0 GBmodel-00103-of-00108.safetensors 5.0 GBmodel-00104-of-00108.safetensors 5.0 GBmodel-00107-of-00108.safetensors 5.0 GBmodel-00016-of-00108.safetensors 5.0 GBmodel-00022-of-00108.safetensors 5.0 GBmodel-00028-of-00108.safetensors 5.0 GBmodel-00034-of-00108.safetensors 5.0 GBmodel-00040-of-00108.safetensors 5.0 GBmodel-00046-of-00108.safetensors 5.0 GBmodel-00052-of-00108.safetensors 5.0 GBmodel-00010-of-00108.safetensors 5.0 GBmodel-00058-of-00108.safetensors 5.0 GBmodel-00064-of-00108.safetensors 5.0 GBmodel-00070-of-00108.safetensors 5.0 GBmodel-00076-of-00108.safetensors 5.0 GBmodel-00082-of-00108.safetensors 5.0 GBmodel-00088-of-00108.safetensors 5.0 GBmodel-00002-of-00108.safetensors 5.0 GBmodel-00008-of-00108.safetensors 5.0 GBmodel-00005-of-00108.safetensors 5.0 GBmodel-00006-of-00108.safetensors 5.0 GBmodel-00007-of-00108.safetensors 5.0 GBmodel-00004-of-00108.safetensors 5.0 GBmodel-00094-of-00108.safetensors 5.0 GBmodel-00100-of-00108.safetensors 5.0 GBmodel-00106-of-00108.safetensors 5.0 GBmodel-00099-of-00108.safetensors 5.0 GBmodel-00093-of-00108.safetensors 5.0 GBmodel-00001-of-00108.safetensors 5.0 GBmodel-00087-of-00108.safetensors 5.0 GBmodel-00057-of-00108.safetensors 5.0 GBmodel-00063-of-00108.safetensors 5.0 GBmodel-00069-of-00108.safetensors 5.0 GBmodel-00075-of-00108.safetensors 5.0 GBmodel-00081-of-00108.safetensors 5.0 GBmodel-00045-of-00108.safetensors 5.0 GBmodel-00051-of-00108.safetensors 5.0 GBmodel-00033-of-00108.safetensors 5.0 GBmodel-00039-of-00108.safetensors 5.0 GBmodel-00021-of-00108.safetensors 5.0 GBmodel-00027-of-00108.safetensors 5.0 GBmodel-00015-of-00108.safetensors 5.0 GBmodel-00003-of-00108.safetensors 5.0 GBmodel-00009-of-00108.safetensors 5.0 GBmodel-00105-of-00108.safetensors 5.0 GBmodel-00108-of-00108.safetensors 3.5 GBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo cerebras/GLM-4.6-REAP-268B-A32B ./model-folder