MiniCPM-V 4.7 35B-A3B NVFP4 — GGUF
NVFP4 GGUF conversions derived from the MiniCPM-V 4.7 35B-A3B model, providing highly efficient 4-bit floating-point quantization for multimodal tasks.
This repository provides NVFP4 (NVIDIA 4-bit floating-point) GGUF files, making the model usable with llama.cpp and other GGUF-compatible inference engines.
Model Overview
MiniCPM-V 4.7 35B-A3B is the first MoE (Mixture-of-Experts) model in the MiniCPM-V series. It is designed to advance the performance–efficiency Pareto frontier, delivering high capability density and smooth on-device inference for resource-constrained environments.
Model Architecture
| Property | Value |
|---|---|
| Architecture | MiniCPM-V 4.7 (MoE) |
| Parameters | 35B-A3B |
| Capabilities | Text, Image, Video |
| Key Technique | Intra-ViT early compression & MoE Architecture |
| Token Compression | Supports mixed 4x/16x visual token compression |
| Supported Modalities | Text, Image, Video |
| Context Length | High-efficiency multimodal context |
GGUF Files
| File | Format | Description |
|---|---|---|
minicpm-v-4.7-35b-a3b-nvfp4.gguf |
NVFP4 | NVFP4 quantized model - optimized for efficiency |
mmproj.gguf |
— | Multimodal projector (vision/video) |
About NVFP4: NVFP4 is NVIDIA's 4-bit floating-point format (E2M1). It offers significant memory savings while preserving quality close to bfloat16, and is accelerated on NVIDIA Blackwell GPUs via llama.cpp.
Usage
llama.cpp (CLI)
# Multimodal inference (Image/Video)
./llama-cli \
-m minicpm-v-4.7-35b-a3b-nvfp4.gguf \
--mmproj mmproj.gguf \
-p "Describe this image in detail." \
--temp 1.0 --top-k 64 --top-p 0.95
llama-server (OpenAI-compatible API)
# Multimodal server
./llama-server \
-m minicpm-v-4.7-35b-a3b-nvfp4.gguf \
--mmproj mmproj.gguf \
--host [IP_ADDRESS] --port 8080
Generation Parameters
Recommended parameters for MiniCPM-V 4.7:
| Parameter | Value |
|---|---|
| Temperature | 1.0 |
| Top-K | 64 |
| Top-P | 0.95 |
Key Features
- Efficiency: Advances the performance–efficiency Pareto frontier for MLLMs.
- Architecture: First open-source MoE model in the series with early visual token compression.
- On-Device Ready: Specifically optimized for smooth on-device inference without compromising performance.
- Versatile: Supports single-image, multi-image, and video understanding.