MiniCPM-V-4.7-35B-A3B-NVFP4
NVFP4 checkpoint of openbmb/MiniCPM-V-4.7-35B-A3B, quantized by me from the official BF16 checkpoint. This is our own conversion, not an ingest of a third-party NVFP4 quant.
NVFP4 here means W4A16 with FP8 E4M3 per-tensor scales, group size 16, weight only, datafree (no calibration corpus). The MoE feed-forward weights are quantized; everything else stays at full precision.
HF repo: esatapedico/MiniCPM-V-4.7-35B-A3B-NVFP4
The llama.cpp-ready GGUF family (six tiers + mmproj projector), built from this checkpoint, lives in esatapedico/MiniCPM-V-4.7-35B-A3B-NVFP4-GGUF with its own card.
At a glance
| Field | Value |
|---|---|
| Format | compressed-tensors nvfp4-pack-quantized |
| Quantization | W4A16 ("NVFP4A16"), group size 16, FP8 E4M3 scales, weight only |
| Calibration | none (datafree) |
| Quantized | fused routed experts + shared expert feed-forward projections |
| Kept in BF16 | attention (full-attn q/k/v/output and linear-attn qkv/gate), linear-attention SSM path (alpha/beta/out), routers, all norms, token embeddings, lm_head, shared_expert_gate, vision tower |
| Zero | MTP tensors (the model ships none; the upstream DSpark drafter is a separate model and is intentionally excluded) |
| Base | openbmb/MiniCPM-V-4.7-35B-A3B (35B MoE VLM, Apache-2.0) |
Quantized scope in detail
Per layer (40 layers, 256 routed experts each): layers.N.mlp.experts.down_proj and layers.N.mlp.experts.gate_up_proj (fused form), plus shared_expert.{gate,up,down}_proj.weight. After quantization the routed experts are packed per-expert (30,720 packed NVFP4 groups) and the shared expert stays fused (120 modules). Verified: the packed census matches exactly, and every unquantized tensor is byte-identical to the source, including all norms. A known llm-compressor norm-perturbation issue was detected and corrected by restoring source bytes; the byte-identity guard re-ran clean afterwards.
The source checkpoint contains no MTP weights (mtp_num_hidden_layers: 1 in config but no tensors; OpenBMB publishes the DSpark drafter as a separate model). This sidesteps the known "llm-compressor silently drops MTP" failure mode, since there are no MTP tensors to lose.
Checkpoint
- Two indexed safetensors shards: ~20.01 GB + ~3.94 GB (≈24 GB total, down from the 70.4 GB BF16 source: 16 shards, 1,152 tensors). Tensor inventory is identical to the upstream checkpoint (35,212,875,824 elements across the same 1,152 names and shapes; verified). Hugging Face displays ≈21B parameters for this repo because the NVFP4 trunk is stored as packed uint8 blobs (shown as
U8) and the headline figure weights them near their true ~4.5-bit width — a display artifact of the packed format, not a smaller model. config.jsonwithquantization_config(quant_method: compressed-tensors,format: nvfp4-pack-quantized), archMiniCPMV4_7ForConditionalGenerationunder the hood (qwen3_5_moe-style MoE text backbone + SigLIP2-style vision tower with window-attention merger)- Tokenizer and chat template intact, preprocessor/processor configs intact
recipe.yaml: the exact llm-compressor recipe used (datafree, expert patterns + ignores)
Serving
- llama.cpp: use the sibling GGUF repo. A compressed-tensors checkpoint is not directly loadable by llama.cpp; the verified end-to-end path is BF16 → this checkpoint → GGUF (with
convert_hf_to_gguf.py --outtype auto, NVFP4 kept native, GGML type 40). - vLLM / compressed-tensors engines: the distribution format mirrors published compressed-tensors NVFP4 checkpoints. Load should be auto-detected from the config. We have not exercised a vLLM load ourselves. Our verification was the byte-identity guard, the GGUF path, and full quality/speed benches on the GGUF family; treat a fresh vLLM load as untested until tried.
Verified with these tiers (GGUF form): PPL 4.19–4.43 across the ladder, MMLU 80.9–82.3% (n=1000) on two Blackwell 16 GB cards, all quality gates pass. Details and speed measurements in the GGUF family card.
Provenance
Derivative work, Apache-2.0 throughout; nothing here was trained or fine-tuned. Credit:
- OpenBMB for the base model and the DSpark drafter (published separately upstream; linked, never duplicated).
- Red Hat AI / vLLM project for llm-compressor, used for the NVFP4 quantization.
- esatapedico for the conversion, verification (byte-identity guard, preflight, packed census), and benchmarking.
Follow along & support
I post updates on new conversions, benchmarks, and what I'm working on over on my site. Follow along there to keep up with new releases and the work in progress. If you'd like to support more of it, you will find the options there. I do this on consumer hardware and like seeing how far it goes. More is on the way.
🌐 esatapedico.com. Updates, work-in-progress, and ways to support.
Note on this card
This model card was written by an AI assistant at the request of the repository author, who did the engineering. As with any AI-generated text, there may be errors; please verify anything important (sizes, configs, commands) against the checkpoint files before relying on it.