mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit-REAP-19B
Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. All OptiQ quants · Docs
19B parameters at 13.5 GB, down from 35B at 24.4 GB, trading Capability Score 87.61 for 81.91.
Half the routed experts are gone. Each of the 40 language layers kept 128 of its 256 experts; top-8 routing is unchanged, so every token still activates 8 experts plus the shared one. The retained experts were copied bit-for-bit from the parent quant, nothing re-quantized or retrained. The cost is uneven: MMLU falls from 86.2 to 63.8, a 22.4 point drop, while GSM8K holds at 90.0, IFEval at 79.9, BFCL at 92.0, and HumanEval at 87.8. Knowledge questions suffer; skills largely survive. Pick it over the parent on a 24 GB Mac, where the parent doesn't fit in memory; pick the parent if knowledge questions matter.
| Parent | This model | ||
|---|---|---|---|
| Language tower on disk | 24.4 GB | 13.5 GB | −45% |
| Vision sidecar | 1.1 GB | 1.1 GB | |
| Parameters | 35.2B | 18.9B | −46% |
| Capability Score | 87.61 | 81.91 | −5.70 |
It works on image and text
Both ran through this quantized model with optiq serve, MLX only:
Image (the words "OPTIQ 42" on a white background, "What text is in the image?"):
OPTIQ 42
Text ("What is the capital of France? One word."):
Paris
What pruning costs
| Benchmark | Parent | This model | Δ |
|---|---|---|---|
| MMLU | 86.2 | 63.8 | −22.4 |
| GSM8K | 94.6 | 90.0 | −4.6 |
| IFEval | 85.0 | 79.9 | −5.1 |
| BFCL-V3 | 93.0 | 92.0 | −1.0 |
| HumanEval | 90.9 | 87.8 | −3.1 |
| HashHop | 76.0 | 78.0 | +2.0 |
| Capability Score | 87.61 | 81.91 | −5.70 |
Details
| Property | Value |
|---|---|
| Parent | mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit |
| Experts retained | 128 of 256 per layer, every layer |
| Active experts per token | 8 + 1 shared (unchanged) |
| Selection | REAP, conditional mean of router weight × expert output norm |
| KL vs the unpruned parent | 0.077 |
| Calibration | optiq six-domain mix, 8 samples; expert coverage min 95.3%, median 100% |
| Vision tower | bf16 sidecar, unchanged from the parent |
| KV cache | kv_config.json, mixed 4/8-bit on the full-attention layers |
Run it
MiniCPM-V 4.7 needs OptiQ 0.5.25 or later. Stock mlx-lm does not know the minicpmv4_7 architecture, so import optiq registers it, and OptiQ loads the vision sidecar.
pip install "mlx-optiq>=0.5.25"
For image input, serve it with an OpenAI- and Anthropic-compatible endpoint:
optiq serve --model mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit-REAP-19B
For the measured mixed-precision KV cache, fetch kv_config.json from this repo and pass it in:
hf download mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit-REAP-19B kv_config.json --local-dir .
optiq serve --model mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit-REAP-19B --kv-config kv_config.json
Then send an image as image_url content. Text-only generation also loads directly:
import optiq # registers minicpmv4_7 and the vision sidecar
from mlx_lm import load, generate
model, tok = load("mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit-REAP-19B")
msgs = [{"role": "user", "content": "Explain why the sky is blue."}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt=prompt, max_tokens=512))
Method
Expert pruning follows REAP (Cerebras Research, ICLR 2026, REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression). OptiQ applies it in the quantized domain, directly on the quantized checkpoint with no bf16 parent, via optiq prune-experts. See the pruning docs.
Links
- Project website: mlx-optiq.com
- Family guide: mlx-optiq.com/docs/minicpm
- All OptiQ quants: mlx-optiq.com/models
- Base model: openbmb/MiniCPM-V-4.7-35B-A3B