DSpark draft model (NVFP4A16 trunk, Q8_0 heads) for MiniCPM-V-4.7-35B-A3B

HF repo: esatapedico/MiniCPM-V-4.7-35B-A3B-DSpark-NVFP4A16 (this file). Parent model files: esatapedico/MiniCPM-V-4.7-35B-A3B-NVFP4-GGUF. NVFP4 safetensors checkpoint: esatapedico/MiniCPM-V-4.7-35B-A3B-NVFP4.

A single-file GGUF of the DSpark speculative-decoding draft model for openbmb/MiniCPM-V-4.7-35B-A3B (Apache-2.0): a 5-layer dense drafter (Qwen3DSpark, hidden size 2048, 8 query heads / 2 KV heads) trained by OpenBMB to pair with MiniCPM-V-4.7-35B-A3B and its tokenizer, proposing up to 15 tokens per forward pass. Converted here from the BF16 checkpoint (openbmb/MiniCPM-V-4.7-35B-A3B-DSpark): NVFP4A16 trunk with Q8_0 heads.

Follow along & support

I post updates on new conversions, benchmarks, and what I'm working on over on my site. Follow along there to keep up with new releases and the work in progress. If you'd like to support more of it, you will find the options there. I do this on consumer hardware and like seeing how far it goes. More is on the way.

🌐 esatapedico.com. Updates, work-in-progress, and ways to support.

Disclaimer: read before downloading

In our measurements this draft runs within about 2% of the upstream Q8_0 draft at equal settings (109.6 vs 112.3 tok/s at budget 5; 77.2 vs 79.5 at budget 15) with effectively tied acceptance (166 vs 167 accepted per 434 generated at budget 5; 178 vs 181 per ~1100 at budget 15), at 0.72x the file size (304 MB vs 425 MB). We also tested an all-NVFP4 pass with BF16 heads (443 MB): it ran 12-21% slower at every setting with effectively the same acceptance, so we settled on Q8_0 heads for the published file.

That is one harness on one hardware shape, not an exhaustive study. Speculative decoding is sensitive to prompt shape, batch size, context length, KV cache precision, offload split, and the parent file in use. None of those axes were swept. If your setup differs, this file may behave better for you. It is published as a measured data point, not as the fastest option: the conversion recipe is verified and the numbers are on the table. As a daily driver the upstream Q8_0 remains the fastest draft we measured; take this file when about 120 MB of download and disk matters more than the last 2% of draft speed.

The file

• dspark-MiniCPM-V-4.7-35B-A3B-NVFP4A16-q8heads.gguf (303,989,824 bytes: 304.0 MB decimal, 289.9 MiB)
• Architecture dflash, 5 blocks, context length 262144, vocabulary 248144, chat template intact (7250 chars), tokenizer with 248144 entries
• Tensor census (97 tensors): 35 NVFP4 (trunk projections) + 3 Q8_0 (vocab-sized heads: markov_w1, markov_w2, fc) + 58 F32 (norms, biases, scales) + 1 BF16
• Metadata: general.name = file basename, general.description set, general.license = apache-2.0, general.file_type = 7 (quantizer-set, advisory; per-tensor types are authoritative)
• Conversion source (not in this repo): the 789 MB BF16 baseline GGUF converted from the upstream BF16 checkpoint.

SHA-256

72dcef2cacb6121e5a8545739eda9c479b01b29778626b9fab31d2f6718b6127  dspark-MiniCPM-V-4.7-35B-A3B-NVFP4A16-q8heads.gguf

Measurements

Parent file MEDIUM (MiniCPM-V-4.7-35B-A3B-NVFP4-MEDIUM.gguf from the GGUF family repo), greedy, n_predict = 256, short chat prompts, -ngl 99 -ngld 99 -c 4096, f16 KV; warm-up excluded, median of 3 repetitions per row. The no-draft, upstream Q8_0, and old-file n15 rows were measured twice (back to back on a busy box, then in a dedicated GPU window with every other GPU workload scaled to zero) with byte-identical greedy acceptance counts across runs; the n5 rows and both q8heads rows are median-of-3 from the dedicated window.

Read: at equal budget this draft accepts within a few tokens of the upstream Q8_0 draft per thousand generated (178 vs 181 at n15; 166 vs 167 at n5) while decoding 2-3% slower, at 0.72x the file size. An all-NVFP4 trial with BF16 heads ran 12-21% slower at the same acceptance, which is why the published file uses Q8_0 heads. The upstream Q8_0 draft at --spec-draft-n-max 5 remains the fastest configuration measured; the base model has no MTP head.

The upstream drafts live in openbmb/MiniCPM-V-4.7-35B-A3B-DSpark-gguf (recommended Q8_0 / Q6_K).

Usage

llama-server \
  -m MiniCPM-V-4.7-35B-A3B-NVFP4-MEDIUM.gguf \
  --mmproj mmproj-MiniCPM-V-4.7-35B-A3B.gguf \
  -md dspark-MiniCPM-V-4.7-35B-A3B-NVFP4A16-q8heads.gguf \
  --spec-type draft-dspark --spec-draft-n-max 5 \
  -ngl 99 -ngld 99 -fa on -c 8192 --jinja

How this was made

1. BF16 baseline. The upstream BF16 checkpoint was converted with convert_hf_to_gguf.py to a BF16-baseline GGUF (arch dflash, 5 blocks). Names and dims verified 62/62 against the source; dtype census 62 BF16.
2. NVFP4A16 conversion. convert_hf_to_gguf.py --outtype auto from the same source kept the 35 trunk tensors as native NVFP4 (GGML type 40) and left norms, biases, and small vectors in F32/BF16: 97 tensors = 35 NVFP4 + 58 F32 + 4 BF16.
3. Q8_0 heads. A per-tensor map (scripts/gguf_tensor_type_map.py: all 97 tensors pinned to current type except markov_w1/markov_w2/fc, which go to q8_0) applied with positional q8_0 and no --allow-requantize, so pinned tensors copy verbatim. Note: positional COPY silently ignores the tensor-type file (found by dry-run); positional q8_0 honors it.
4. Verification. Every tensor data region of the final file was hash-compared against the all-NVFP4 trial: 94/97 identical, only the 3 heads changed (BF16 to Q8_0, dims unchanged). One cosmetic header difference: conf_proj.weight changed shape spelling from (2304, 1) to (2304) (same 2304 elements, same bytes; the quantizer squeezes the trailing size-1 dim when rewriting headers). Census 35 NVFP4 + 3 Q8_0 + 58 F32 + 1 BF16.

Needs a llama.cpp build with MiniCPM-V 4.7 support (merged upstream 2026-10-10, PR #29416) plus the NVFP4 CUDA kernels for Blackwell (sm_120). Mainline is enough, no fork needed.

Attribution & provenance

Derivative work built from Apache-2.0 artifacts. Nothing here was trained. Credit belongs to:

1. OpenBMB for the base model and the DSpark drafter checkpoint this file converts.
2. llama.cpp contributors for the GGUF toolchain, the NVFP4 path with Blackwell kernels, and the MiniCPM-V 4.7 conversion/serving support.
3. esatapedico for the conversion, verification, and benchmarking.

"MiniCPM-V" is a name of OpenBMB. This repository is not affiliated with, sponsored by, or endorsed by OpenBMB.

License

Apache-2.0, identical to every upstream artifact. The base model license governs; GGUF conversion and quantization are transformations, not new training. When redistributing, please retain attribution to OpenBMB (base model) as above.

Note on this card

This model card was written by an AI assistant at the request of the repository author, who did the engineering. As with any AI-generated text, there may be errors; please verify anything important (hashes, sizes, commands) against the file itself before relying on it.