MiniCPM-V 4.7 35B-A3B — Attention8 + BF16 recurrence

Benchmark-selected mixed affine MLX quantization of MiniCPM-V 4.7, with a qualified MTPLX adapter and publisher DSpark speculative decoding. The complete artifact is 22.17 GiB. This release preserves the original tokenizer, chat template, image processor, and BF16 vision weights. It does not edit model behavior or perform fine-tuning.

Quantization and selection

Converted directly from the publisher BF16 parent.

452 modules are quantized: 236 at 8-bit and 216 at 4-bit. The recurrent input projection weights remain BF16; the original float32 recurrence accumulation is preserved. All 2,096 stored arrays passed exact reconstruction parity against the source-derived conversion. Loaded-model versus layer-offload log probabilities matched exactly on eight text cases.

The selected quant measured 0.000834 nats mean KL against the BF16 parent on 11 teacher-forced text, OCR, spatial, and context cases. Mean NLL change was −0.01721 on this small suite. These measurements indicate low distortion on the evaluated inputs; they do not establish universal fidelity. Faster MXFP4 candidates were rejected for a held-out logic regression. A 5-bit expert candidate was slower and had higher measured KL. Additional packed rounding did not meet the 5% speed-improvement threshold.

Run with MTPLX and DSpark

Requires an Apple Silicon Mac. Qualification used Python 3.11.14 on an Apple M4 Max with 64 GiB unified memory. Allow space for the 22.17 GiB target, a roughly 0.73 GiB draft, runtime dependencies, and caches. This bundle installs into its own virtual environment.

Download this repository with the Hugging Face CLI, then bootstrap from the downloaded folder:

hf download Shiftedx/MiniCPM-V-4.7-35B-A3B-Attention8-BF16Recurrence-MLX-MTPLX --local-dir MiniCPM-V-4.7-Attention8
cd MiniCPM-V-4.7-Attention8
python3.11 runtime/bootstrap.py
runtime/.venv/bin/python runtime/serve_minicpm.py --engine dspark --model . --model-id minicpm-v4.7-35b-a3b --port 18147

The launcher downloads the separate publisher DSpark draft at that immutable revision. For offline use, provide its snapshot with --draft /path/to/draft. For native MTPLX autoregressive decoding, use --engine mtplx-ar.

The local OpenAI-compatible endpoint is http://127.0.0.1:18147/v1; the server binds to loopback and handles requests serially. Example:

curl http://127.0.0.1:18147/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"minicpm-v4.7-35b-a3b","messages":[{"role":"user","content":"What is 6 multiplied by 7?"}],"temperature":0,"max_tokens":64}'

Images use OpenAI image_url content parts containing base64 data:image/... URLs. Multiple images, tools/tool replies, Unicode, and SSE streaming were qualified. HTTP video inputs are not implemented; video qualification used the MLX-VLM SDK. Run runtime/.venv/bin/python runtime/video_example.py /path/to/video.mp4 for SDK video inference. enable_thinking defaults to false, matching the benchmark contract. DSpark is enabled for temperature-zero requests with block size eight; sampled requests use MTPLX AR.

Runtime contract: MLX 0.32.2, MLX-LM 0.31.3, Transformers 5.14.1, MTPLX 2.12.2 (commit), and MLX-VLM 0.7.8 (commit). Bootstrap applies both bundled correctness patches to those exact source trees. The overlay provides MiniCPM model registration; stock unpatched MTPLX alone is insufficient. See upstream contribution notes.

Weights are materialized on CPU and inference runs on GPU. Settings enable sustained prefill in 512-token chunks, suppress full-prefill vocabulary logits, and cap the MLX allocator cache at 2 GiB. KV caches retain the runtime's original precision; no KV quantization was enabled.

No native MTP: the parent config advertises MTP but its checkpoint contains no MTP tensors. This artifact declares zero native MTP layers. DSpark is a separate draft/verifier engine supplied by pinned MLX-VLM, operating on the MTPLX-loaded target; it is not MTPLX's native MTP engine.

Local qualification benchmarks

These are pre-publication measurements on the exact staged weights, not the standardized post-publication Shiftedx Bench gate. Aggregate data and contract are in BENCHMARKS.json; provenance and precision metadata are in QUALIFICATION.json.

The hard suite used doubled output budgets; JSON Pointer and weighted scheduling needed four-times budgets to eliminate truncation. Validators were unchanged. The remaining failures were deterministic_toposort_layers and ttl_lru_cache. BF16 full-generation comparison was not performed for these failures, so their origin is not established.

The warm DSpark workload mix contains short counting, code, and long-prompt generation; individual workload medians were roughly 200, 115, and 192 tokens/s. Draft acceptance and latency depend strongly on the workload. These are warm results, excluding model loading; API throughput includes request processing and prefill. Native AR and DSpark measurements are separate runs, not a controlled same-run speedup claim. Peak MLX allocation is not total process/system memory.

The parent advertises 262,144 tokens, but this release has only qualified retrieval through 32,755 prompt tokens. The full advertised window and production concurrency have not been qualified. Do not interpret a small successful fixture suite as a broad capability benchmark.

Integrity, provenance, and license

RELEASE_MANIFEST.json and SHA256SUMS cover the public payload. The manifest and SHA256SUMS describe the immutable weight-bearing revision 5ec49aa0f69e263e41d29ca0c64c4535c83d91ba. Verify that pinned snapshot with shasum -a 256 -c SHA256SUMS. The current README is a later benchmark-only update; its checksum differs by design. Model weights and runtime payload hashes remain unchanged.

Model weights retain the publisher's Apache-2.0 license declaration. Bundled adapter/server code is Apache-2.0. MLX-VLM upstream code is MIT; MTPLX upstream code is Apache-2.0 and is fetched separately with its original notices. See NOTICE.

Shiftedx Bench post-publication qualification

This table was generated from the frozen lightweight quant gate after the model weights were published. Categories remain separate; the benchmark does not produce a composite intelligence score.

• Tested model revision: 5ec49aa0f69e263e41d29ca0c64c4535c83d91ba
• Benchmark: Shiftedx Bench v0.5.1
• Context lengths represented: 4,096, 16,384, 65,536, 131,072 prompt tokens; effective tested context: 4,096 tokens
• Runtime contract: MTPLX 2.12.2 patched adapter; DSpark greedy block8, sampled AR; thinking off; KV cache off; MTP depth not-applicable
• Host: Apple M4 Max / 64 GiB
• Total measured request wall time: 451.31 seconds
• 260,096-token status: not run; it is outside the lightweight quant gate.

Scores are specific to the linked model revision, benchmark revision, runtime contract, and host. Changing weight precision, KV-cache precision, reasoning mode, template, or speculative depth creates a different benchmark candidate.