MiniCPM-V-4.6 · Huihui-abliterated · 8-bit (gs64) · downsample 4x (MLX, tuned for oMLX)

English · 中文

A small, fast vision model: OCR, charts, screenshots, multi-image comparison at ≈ 140–180 tok/s in only 2.3 GB. This build is the abliterated (refusal-removed) Huihui model converted to 8-bit MLX, with the image-token downsample mode changed from 16x to 4x — the setting that made large-page OCR complete on our tests.

⚠️ Abliterated model — for emergency / red-team / research use in an OS-level sandbox. It is a vision specialist: plain-text history Q&A is not its job (it dated one historical event wrongly in our probe), and counting can loop. Use a larger model for those.

At a glance

Item	Value
Architecture	MiniCPM-V 4.6 (vision encoder + LLM)
Quantization	8-bit, group size 64, affine on the language-model layers; the rest FP16
Size	2.3 GB; ~2.3 GB resident
Context	65536 configured; max output 16384
Speed (M1 Ultra)	≈ 140–180 tok/s decode; 35-item vision suite in 12–14.5 s; 30-line dense page in 6.2–7.7 s
Thinking	Off by default; `chat_template_kwargs: {"enable_thinking": true}` turns it on
Default sampling	temperature 0 (greedy), top_p 1.0, top_k 0, repetition_penalty 1.0
License	Apache-2.0 (upstream OpenBMB MiniCPM-V-4.6 and Huihui repos are Apache-2.0)

Lineage: openbmb/MiniCPM-V-4.6 → huihui-ai/Huihui-MiniCPM-V-4.6-abliterated (rev 1c0ce8ae…, SHA-256 verified against the official LFS hashes) → mlx_vlm convert -q --q-bits 8 --q-group-size 64 --q-mode affine → downsample_mode: 4x in config.json, preprocessor_config.json and processor_config.json (all three must agree) + processor_config.json / video_preprocessor_config.json image-size limits kept from our previous production build.

Measured results

Metric	Previous (mlx-community 8bit)	Heretic abliterated	This (Huihui)
Vision suite (35 items)	35/35	33/35 (spatial descriptions degraded)	35/35
Large-page OCR (30 lines, ×3 rounds)	120/120, 0 missed	120/120	120/120, 0 missed, 144 tok/s (previous 145)
Sensitive probes answered	4/4 (already low-refusal)	4/4	4/4

We swapped for stack consistency (every chat/vision model uncensored) without losing OCR accuracy or speed. Heretic was rejected for worse spatial descriptions.

Why 4x downsample

The upstream default 16x merges too many patches for dense pages. With 4x the large dense-page test went to 120/120 and the 35-item suite to 35/35 (two zero-variance runs), at no measurable speed cost on our tests.

Routing in our stack

Task	Use
OCR, receipts, chart reading, descriptions, multi-image compare	this model (3–4× faster than the 27B, ~1/9 memory)
Counting, spatial reasoning, vision that needs thinking	the Qwen3.8-27B build

We also tried FP16 casting of this model: only ≈ 3 % faster → not adopted.

Recommended oMLX settings

omlx/model_settings.entry.json (key = directory name):

{
  "max_context_window": 65536, "max_tokens": 16384,
  "temperature": 0.0, "top_p": 1.0, "top_k": 0, "repetition_penalty": 1.0,
  "mtp_enabled": false, "ttl_seconds": 600, "model_alias": "MiniCPM-V-4.6"
}

Image limits in the oMLX server settings: max_image_upload_size: 50MB, max_image_side_length: 2048. Never set model_type_override for vision models.

Quick start

HF_ENDPOINT=https://hf-mirror.com hf download YCF-AI/MiniCPM-V-4.6-Huihui-abliterated-8bit-MLX \
  --local-dir ~/models/MiniCPM-V-4.6-8bit
# oMLX: add ~/models to model_dirs, merge the settings entry (key = directory name), restart.
# mlx-vlm: python -m mlx_vlm generate --model ~/models/MiniCPM-V-4.6-8bit --image page.png --prompt "OCR this page" --temperature 0

The mlx-vlm line is the standard interface for this model class; we run it through oMLX and did not test the CLI separately.

Rebuild recipe

python -m mlx_vlm convert --hf-path <huihui-src-dir> --mlx-path <out> -q --q-bits 8 --q-group-size 64 --q-mode affine
# then set "downsample_mode": "4x" in config.json, preprocessor_config.json, processor_config.json

Verify afterwards: 35-item vision suite and a dense-page OCR with line-by-line ground truth; purge SSD KV cache entries of this model name if weights changed under the same ID.

Known limits

Counting can enter a loop; spatial reasoning is average → use the 27B.
Text-only factual Q&A is unreliable (not its role).
Measured on Mac Studio M1 Ultra 64 GB, macOS 15.8, oMLX 0.7.0, mlx-vlm 0.7.1; one machine.

中文说明

English · 中文

小而快的视觉模型：OCR、图表、截图、多图对比，仅 2.3 GB，解码约 140–180 tok/s。本版是去审查（abliterated）的 Huihui 模型转成 8-bit MLX，并把图像 token 的 downsample 模式由 16x 改为 4x——这一改动让大图 OCR 在我们的测试中完整无漏。

⚠️ 去审查模型——仅用于应急 / 红队 / 研究，须放在 OS 级沙箱。它是视觉专用：纯文本史实问答不是它的职责（探针中曾把一个历史事件日期答错），计数可能循环；这些请交给更大的模型。

概览

项目	值
架构	MiniCPM-V 4.6（视觉编码器 + LLM）
量化	8-bit、group size 64、affine（语言模型），其余 FP16
体积	2.3 GB，常驻约 2.3 GB
上下文 / 最大输出	65536 / 16384
速度（M1 Ultra）	解码约 140–180 tok/s；35 项视觉套件 12–14.5 s；30 行大图 6.2–7.7 s
思考	默认关；`chat_template_kwargs: {"enable_thinking": true}` 开启
默认采样	temperature 0（greedy）、top_p 1.0、top_k 0、repetition_penalty 1.0
许可	Apache-2.0（上游 OpenBMB 与 Huihui 仓库均为 Apache-2.0）

血统： openbmb/MiniCPM-V-4.6 → huihui-ai/Huihui-MiniCPM-V-4.6-abliterated（rev 1c0ce8ae…，已按官方 LFS 哈希校验）→ mlx_vlm convert -q --q-bits 8 --q-group-size 64 --q-mode affine → 在 config.json、preprocessor_config.json、processor_config.json 三处同步设 downsample_mode: 4x，并保留此前生产版的图像尺寸上限。

实测结果

指标	旧版（mlx-community 8bit）	Heretic 去审查	本版（Huihui）
视觉 35 项	35/35	33/35（空间方位描述退化）	35/35
大图 OCR（30 行 ×3 轮）	120/120 漏 0	120/120	120/120 漏 0，144 tok/s（旧 145）
敏感探针	4/4（本就低审查）	4/4	4/4

换版是为了全栈一致去审查，同时不损失 OCR 精度与速度；Heretic 因空间描述变差弃用。

为什么用 4x：上游默认 16x 对密集页面合并了过多 patch；改 4x 后大图 120/120、35 项 35/35（两次零方差），速度无可测损失。

本机路由：OCR、票据、图表读数、描述、多图对比 → 本模型（比 27B 快 3–4 倍、内存约 1/9）；计数、空间推理、需要思考的看图任务 → Qwen3.8-27B 版。我们也测过本模型 FP16 化：仅快约 3 %，不采用。

快速开始与复现

下载命令见上方（国内用 HF_ENDPOINT=https://hf-mirror.com 直连）。复现：python -m mlx_vlm convert --hf-path <huihui 源目录> --mlx-path <输出> -q --q-bits 8 --q-group-size 64 --q-mode affine，再在三处配置里把 downsample_mode 设为 4x；之后跑 35 项视觉套件和带逐行标注的密集页 OCR 验证；同一模型 ID 下权重变化须清理该模型的 SSD KV 缓存。mlx-vlm 命令行为该类模型的标准接口，我们实际通过 oMLX 运行，未单独测试 CLI。

已知限制

计数可能循环、空间推理一般 → 用 27B；纯文本史实问答不可靠。
数据来自 Mac Studio M1 Ultra 64 GB、macOS 15.8、oMLX 0.7.0、mlx-vlm 0.7.1，仅一台机器。

YCF-AI/MiniCPM-V-4.6-Huihui-abliterated-8bit-MLX