MiniCPM-V-4.7-35B-A3B for mlx-serve (4-bit)
An MLX affine quantization of openbmb/MiniCPM-V-4.7-35B-A3B, distributed under the base model's Apache-2.0 license.
Text-only pack
This pack contains language model weights only. It does not support image or video inputs and does not include the MiniCPM vision tower or merger. Its native root model type is minicpmv4_7, and it requires mlx-serve v26.10.2 or later.
Quantization
Affine 4-bit quantization, group size 64. Routed MoE experts are 4-bit; per-layer router gates and shared-expert gates are 8-bit. Norms, scales, and biases remain BF16.
Setup
mlx-serve serve --model pgmharikrishnan/MiniCPM-V-4.7-35B-A3B-MLX-Serve-4bit
Benchmark
Standard suite on Apple M4 Max with mlx-serve 26.10.2:
| Benchmark | Result |
|---|---|
| MMLU-Pro (400 questions) | 89.75% (359/400) |
| GSM8K (50 questions) | 100% |
| Coding (6 tasks) | 6/6 |
| Decode | 60.6 tok/s |
| Prefill | 343 tok/s |
| Peak memory | 24.7 GB |
MMLU-Pro is public, so training contamination cannot be ruled out.
Attribution
MiniCPM-V 4.7 was created by OpenBMB. This is an independent MLX conversion for mlx-serve.
Disclaimer
This is an unofficial, text-only conversion. Validate it for your workload before production use.