MiniCPM-V-4.7-1B — Standalone Native Vision Bundle
Visual weights extracted directly from openbmb/MiniCPM-V-4.7-1B, fixed to revision 286817f1132b998913a129a221c89b4402b4ab80. Includes the 27-layer visual tower, its internal window-attention insert merger, and the external MLP merger projecting to 1024 dimensions. Both 4x and 16x paths are retained.
Only visual/connector weights are included. No language-model weights, language-model head, or additional training. This bundle uses the original Transformers visual classes through the included explicit loader; it is not a timm remap and does not register an AutoModel remote-code implementation.
Contents and boundaries
The vision_tower includes the internal early-compression module. The merger namespace is the external projection. Skipping the internal module in 4x mode follows the upstream implementation. Target grids must satisfy the source merger divisibility requirements.
model.safetensors: 459 tensors, 548,266,416 parameters across the selected visual modules, stored as BF16; 1.097 GB.source_config.json: complete source configuration for correctly constructing visual components; its text configuration is metadata only.vision_config.json: source vision subconfiguration.config.json: bundle provenance and loader metadata, not a full VLM configuration.tensor_mapping.json: exact source keys, original shard offsets, and output keys.vision_model.py: local strict loader; constructs visual modules only.extract_multimodal_vision.py,validate_multimodal_vision.py: reproducible extraction and verification scripts.extraction.json,validation.json: file checksum, provenance and test results.- Source preprocessing configuration files where supplied, plus
SOURCE_README.mdand license/attribution files.
Download and load
pip install -r requirements.txt
from huggingface_hub import snapshot_download
import sys
root = snapshot_download('gatilin/minicpm_v4_7_1b_vision')
sys.path.insert(0, root)
import torch
from vision_model import load_model
model = load_model(root)
# Packed NaViT patch layout: 4x4 patches from one synthetic image.
pixel_values = torch.randn(1, 3, 14, 16 * 14)
target_sizes = torch.tensor([[4, 4]], dtype=torch.int32)
with torch.inference_mode():
result = model(pixel_values, target_sizes, downsample_mode='16x')
print(result.last_hidden_state.shape) # [1, 4, 1152]
print(result.pooler_output[0].shape) # [1, 1024]
The example uses synthetic, already-packed visual input tensors to show the exact native API. For real images/videos, use the upstream processor at the pinned source revision and preserve its resize, normalization, patch order, temporal padding, and grid metadata. Raw BCHW images cannot be passed directly to the flattened/packed API. This repository does not include a tokenizer or text-generation pipeline. load_model(..., dtype=torch.bfloat16, device='cuda') is available for compatible hardware; FP32 CPU is the tested validation setting.
Verification
Every exported tensor's shape, dtype and raw bytes were compared with the downloaded exact source byte ranges: all identical. Only source key prefixes were removed; no reshaping, quantization, retraining, or numerical conversion was applied to stored weights.
Both strict state loading and 6 native-feature comparisons passed. Reference outputs came from the original Transformers parent's get_image_features / applicable get_video_features methods, with weights reconstructed directly from source ranges; unused language-model modules were not allocated. Tests covered square and rectangular patch grids, packed multiple-image inputs, and both 4x and 16x compression paths. Maximum absolute errors: hidden features 0, projected features 0. Tolerance: rtol=1e-5, atol=1e-5.
A native-input 448×448 resolution smoke test also passed, producing finite outputs. Tests used torch 2.14.1, Transformers 5.20.1, FP32 CPU, SDPA, seed 17. These are extraction and visual API consistency tests, not end-to-end real-image processing, language generation, task benchmarks, or GPU/BF16 accuracy claims.
Reproduce
python extract_multimodal_vision.py minicpm_v4_7_1b --output ./reproduced --refresh-source
python validate_multimodal_vision.py ./reproduced/minicpm_v4_7_1b
The extractor uses Python's standard library and validated HTTP ranges to download only selected visual tensors. --refresh-source refills the resumable .download ranges for source-byte revalidation. The validation script imports vision_model.py from the same directory as itself. The published bundle excludes download caches.
中文说明
本仓库保留原生视觉塔及其训练过的压缩/投影模块，通过随附的 vision_model.py 加载，不会构造语言模型。已完成逐张量源字节核验、严格加载、原生视觉特征与投影输出对比，以及 448×448 分辨率前向检查。主权重保持 BF16。输入需要使用上游处理器生成的原生 patch 排列和网格信息；这里的示例及验证覆盖原生视觉张量接口，不代表整套实图到文本生成或下游精度测试。
License and source
Follow the upstream model's apache-2.0 license and attribution. Original model documentation is preserved in SOURCE_README.md. This is an independent extraction without affiliation with or endorsement by the original authors. The included loader depends on the upstream Transformers implementation and its license.