mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit-REAP-14B
Built with mlx-optiq, the MLX-native toolkit to quantize, prune, fine-tune, and serve LLMs locally on Apple Silicon. All OptiQ models · Docs
10.4 GB instead of 17.5 GB. 9.4 GB of memory to run instead of 16.4 GB. Capability Score 68.13 against the parent's 75.75.
| Parent | This model | ||
|---|---|---|---|
| On disk | 17.5 GB | 10.4 GB | −40% |
| Peak memory | 16.4 GB | 9.4 GB | −42% |
| Parameters | 25.8B | 14.3B | −45% |
| Capability Score | 75.75 | 68.13 | -7.62 |
50% of the routed experts are removed from mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit; active parameters per token are unchanged, since top-8 routing is preserved and only the stored expert bank shrinks. That is why it gets smaller without getting slower.
Retained experts are copied bit-for-bit from the parent quant. Nothing is dequantized, re-quantized, merged, or retrained.
What pruning costs
| Benchmark | Parent | This model | Δ |
|---|---|---|---|
| MMLU | 65.0 | 38.9 | -26.1 |
| GSM8K | 93.8 | 89.1 | -4.7 |
| IFEval | 73.0 | 80.8 | +7.8 |
| BFCL-V3 | 91.5 | 90.0 | -1.5 |
| HumanEval | 90.2 | 72.0 | -18.2 |
| HashHop | 41.0 | 38.0 | -3.0 |
| Capability Score | 75.75 | 68.13 | -7.62 |
Gemma-4 gives up more than the Qwen models do at the same 50 % retention. It routes over 128 experts where they route over 256, and the divergence from the unpruned parent measures KL 0.56 against their 0.07–0.13. The cost shows up in MMLU first, as it does everywhere, but here HumanEval moves too — on the Qwens procedural ability is essentially untouched.
If that trade is wrong for your workload, the model is not the only option: optiq prune-experts <parent> --retain 96 keeps three quarters of the bank and measures KL 0.33, at a correspondingly smaller saving.
Details
| Property | Value |
|---|---|
| Experts retained | 64 of 128 per layer |
| Active experts per token | 8 (unchanged) |
| Allocation | uniform (64 of 128 in every layer) |
| Size | 10.4 GB (parent 17.5 GB) |
| Parameters | 14.3B (parent 25.8B) |
| Selection | REAP — router weight × expert output norm, weighted by selection frequency (this router renormalizes its weights, so frequency is what carries preference) |
| Calibration | optiq six-domain mix, 8 samples |
| MTP sidecar | absent |
Use it
pip install mlx-optiq
optiq serve --model mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit-REAP-14B
from mlx_lm import load, generate
model, tok = load("mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit-REAP-14B")
print(generate(model, tok, prompt="Hello", max_tokens=64))
Method
Expert pruning follows REAP (Cerebras Research, ICLR 2026 — REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression). Experts are ranked by the conditional mean of router weight × expert output norm over calibration data; the lowest-ranked are removed and the router is sliced to match.
OptiQ applies it in the quantized domain — directly on a quantized checkpoint, with no BF16 parent and no dequantization of survivors — via optiq prune-experts. See the pruning docs.