MiniCPM-V-4.7-35B-A3B-5bit
Converted from openbmb/MiniCPM-V-4.7-35B-A3B at revision 17e651151c161dce119cd0ac57570f60da8a0b40 with MLX-VLM main (e928101a144a3f11a27c7ef8d71b6d889b2d250c, version 0.7.8). Support was added in PR #2499.
5BIT weight quantization (affine, group size 64). This is mixed precision: vision modules, visual mergers, embeddings, output heads, and linear attention remain BF16; recurrent A_log tensors retain their source precision. The 35B-A3B expert routers use 8-bit affine quantization.
See the original model card for the architecture, intended uses, and limitations. This conversion retains the source model's apache-2.0 license.
Usage
Install MLX-VLM from main with MiniCPM-V 4.7 support:
pip install -U git+https://github.com/Blaizzy/mlx-vlm.git@main
mlx_vlm.generate --model nativ-community/MiniCPM-V-4.7-35B-A3B-5bit --image path/to/image.jpg --prompt "Describe this image." --max-tokens 200 --temperature 0
The native NumPy/Pillow processor requires neither PyTorch nor remote code. It supports text, images, multiple images, and video. The default visual downsampling is 16x; pass --processor-kwargs '{"downsample_mode": "4x"}' for finer detail.
Reproduce conversion
mlx_vlm.convert --hf-path openbmb/MiniCPM-V-4.7-35B-A3B --revision 17e651151c161dce119cd0ac57570f60da8a0b40 --mlx-path MiniCPM-V-4.7-35B-A3B-5bit --dtype bfloat16 -q --q-bits 5 --q-group-size 64 --q-mode affine
For the exact converter version, install MLX-VLM at commit e928101a144a3f11a27c7ef8d71b6d889b2d250c. Conversion options, per-module quantization overrides, and weight SHA-256 hashes are recorded in conversion.json.
Validation
The saved checkpoint passed strict reloading, quantization configuration checks, greedy text generation, and cat-image generation in both 16x and 4x downsampling modes on JarvisLabs RTX PRO 6000 Blackwell (cuda) with MLX 0.32.3. See validation.json for the outputs and environment. These are smoke checks, not a quality benchmark. Floating-point 4-bit formats can differ in output quality from affine quantization.