MiniCPM-V-4.7-35B-A3B — GGUF quants (IQ3_S · Q3_K_L · TQ3_4S)

Three GGUF quantizations of openbmb/MiniCPM-V-4.7-35B-A3B, plus the F16 vision projector. 35B MoE with 3B active parameters per token, 256 routed experts (8 used), 256K context.

Which runtime do I need?

Read this first — the three variants do not have the same runtime requirement. Compatibility below was determined by reading each file's tensor-type table, not by assumption. Upstream llama.cpp defines GGML_TYPE_COUNT = 43; any tensor type id ≥ 43 exists only in the llama.cpp-tq3 fork.

TQ3_4S is the TurboQuant four-scale type. It is id 46, outside mainline's type table, so stock llama.cpp will not load that file — it needs turbo-tan/llama.cpp-tq3. If you want a 3–4 bpw quant that runs on plain llama.cpp, pick IQ3_S (smallest) or Q3_K_L.

The architecture itself (qwen35moe) is supported by both, and all three files carry only standard GGUF metadata — no fork-specific KV keys.

Files

Companion files (tokenizer, config, chat template, image preprocessor) are in companion/. mmproj-model-f16.gguf is required for image input — this is a vision-language model, and without the projector it runs text-only.

bpw is measured from each file (weight bytes × 8 ÷ 34,659,889,792 parameters), not taken from the variant label.

Quick start

Text-only (works with mainline llama.cpp for IQ3_S / Q3_K_L):

llama-cli -m iq3_s/MiniCPM-V-4_7-35B-A3B-IQ3_S.gguf -ngl 99 -c 8192

With vision — always pass --mmproj:

llama-server -m q3_k_l/MiniCPM-V-4_7-35B-A3B-Q3_K_L.gguf \
  --mmproj mmproj-model-f16.gguf \
  -ngl 99 -c 8192 --jinja

For tq3_4s/, build the fork first:

git clone https://github.com/turbo-tan/llama.cpp-tq3 && cd llama.cpp-tq3
cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release -j
./build/bin/llama-server -m tq3_4s/MiniCPM-V-4_7-35B-A3B-TQ3_4S.gguf \
  --mmproj mmproj-model-f16.gguf -ngl 99 -c 8192 --jinja

Context length

The files declare context_length = 262144 (256K), which is the architectural maximum from the model config. That is a spec number, not a tested ceiling: 256K of KV cache will not fit in consumer VRAM alongside these weights. On a 16 GB card we measured this family serving about 128K with a quantized KV cache (~100 tok/s decode); past that, decode falls off sharply. Treat 256K as an upper bound requiring KV-cache quantization and enough memory, and size your context to your hardware.

Benchmarks

Pending. Quality numbers for these three variants have not been measured yet, so none are claimed here. The facts on this card that are verified: file structure, tensor types, measured bpw, parameter count, metadata consistency, and source lineage (below).

Provenance

• Official base: openbmb/MiniCPM-V-4.7-35B-A3B
• Source for these quants: the official OpenBMB F16 GGUF from openbmb/MiniCPM-V-4.7-35B-A3B-gguf — MiniCPM-V-4_7-35B-A3B-F16.gguf (69,375,191,040 bytes / 64.61 GiB) and mmproj-model-f16.gguf (1,119,575,840 bytes / 1.04 GiB). Both verified byte-identical to the upstream release. The projector shipped here is that upstream file, unmodified.
• Quants produced from that F16 with llama-quantize; the F16 and projector were not reconverted.
• These three variants are not published upstream. The official GGUF repo ships Q4_0, Q4_1, Q4_K_M, Q4_K_S, Q5_0, Q5_1, Q5_K_M, Q5_K_S, Q6_K, Q8_0 and mix_q3gud_im — no IQ3_S, no Q3_K_L, no TQ3_4S. Our IQ3_S (14.22 GiB) is also slightly smaller than upstream's smallest offering, mix_q3gud_im (14.44 GiB).
• Verified on each shipping file: 733 tensors, GGUF v3, general.architecture = qwen35moe, block_count = 40 with tensors blk.0–blk.39 present, nextn_predict_layers = 0, expert_count = 256, expert_used_count = 8, embedding_length = 2048.
• No importance matrix was used. IQ3_XS is deliberately not offered here: it hard-requires a calibrated imatrix (llama-quantize aborts without one — "the result will be garbage"), and no imatrix exists for this model. Publishing an uncalibrated IQ3_XS would mean shipping a knowingly degraded file.
• Requantizing from GGUF strips the original provenance keys, so the lineage above is established from the official F16 byte-size match rather than from the files' own metadata.

License

Apache-2.0, matching the official OpenBMB release. The upstream repository declares license: apache-2.0 but ships no LICENSE file, so the canonical Apache-2.0 text is included here as LICENSE. Weights remain the property of their authors; this repo distributes quantizations only.