Runtime: Osaurus 0.25.21 or newer (
osaurus.json:required_osaurus_version).
OsaurusAI/MiniCPM-V-4.7-35B-A3B-JANG_4M
A JANG bundle of openbmb/MiniCPM-V-4.7-35B-A3B: a 35B mixture-of-experts model with 3B active parameters (40 layers, GatedDeltaNet linear attention and full attention 3:1, 256 routed experts with 8 active plus a shared expert), with image, multi-image and video understanding, thinking on by default, and Qwen-style XML tool calls. 65.7 GiB of bf16 weights in 23.0 GiB.
What is quantized how
| part | format |
|---|---|
| routed experts gate / up | affine 4-bit, group 64, GPTQ per expert |
| routed experts down | affine 6-bit, group 64, GPTQ per expert |
| shared expert, attention, GatedDeltaNet projections | affine 8-bit, GPTQ |
| embeddings, lm_head | affine 8-bit |
| routers | bf16 |
| vision tower + projector | affine 8-bit (fc2 bf16) |
| norms, conv, A_log, dt_bias | source values |
Every quantized matrix with an input Hessian was rounded with GPTQ. The Hessians come from the model's own outputs: the bf16 model wrote reasoning traces (thinking on and off), tool calls, code, long-document answers and multi-turn agent turns across 11 prompt families at its native sampling settings, and those sequences were the calibration data. Affine quantization is MLX's own grid (bf16 scales and biases), so the bundle loads at full speed on the standard kernels.
Fidelity vs the bf16 model
150,892 teacher-forced positions on 164 held-out sequences the bf16 model generated itself (never seen in calibration), scored against the bf16 model: top-128 KL, top-1 agreement, and the change in log-likelihood of the real next token (dNLL).
| size | KL median | KL mean | top-1 | dNLL | |
|---|---|---|---|---|---|
| 35B-A3B-JANG_4M | 23.0 GiB | 1.9e-07 | 0.0057 | 98.46% | +0.0032 |
By prompt family (KL median, top-1):
| family | KL median | top-1 |
|---|---|---|
| agentic / tool use | 0.0000 | 99.3% |
| coding | 0.0000 | 98.4% |
| creative writing | 0.0020 | 95.7% |
| cybersecurity | 0.0001 | 97.2% |
| long documents | 0.0000 | 98.3% |
| math | 0.0000 | 99.4% |
| networking | 0.0000 | 99.1% |
| office tasks | 0.0000 | 99.0% |
| science | 0.0000 | 99.0% |
| terminal agent | 0.0000 | 99.3% |
| turn after a tool result | 0.0000 | 99.0% |
Decisions: the quantized model picks the same token as bf16 at 80 / 80 </think> closes and at 23 / 25 tool-call starts (bf16 vs itself: 23 / 25).
Speculative decoding
dspark/ holds openbmb's DSpark draft model for this size (openbmb/MiniCPM-V-4.7-35B-A3B-DSpark), quantized to affine 8-bit (its Markov and confidence heads stay bf16). It is stamped on by default (jang_config.json → speculative). Draft acceptance has not been measured on the 35B; on the 1B drafter the 8-bit quantization did not change acceptance (2.74 vs 2.74 tokens per round, greedy).
Model notes
- Modalities: text, images (multi-image, high-resolution slicing), video. No audio.
downsample_mode16x (default) or 4x selects the visual token budget. - Reasoning: on by default; the chat template opens
<think>.enable_thinking: falseturns it off. There are no reasoning-effort levels. - Tools: Qwen3-Coder-style XML (
<tool_call><function=…><parameter=…>). - Sampling (
generation_config.json, mirrored injang_config.json): temperature 1.0, top-p 0.95, top-k 20, EOS<|im_end|>/<|endoftext|>. - The source config lists one MTP layer, but the checkpoint contains no MTP weights; none is included.
Source
openbmb/MiniCPM-V-4.7-35B-A3Brevision4c7623a9326e, Apache-2.0.- Quantization and bundle by Jinho Jang (eric@osaurus.ai), 2026-10-12.