DeepSeek ViT 412M — V4.1 Flash 1017

Standalone vision backbone extracted from DeepSeek-V4.1-Flash-1017, converted to native timm format without additional training. No language-model weights or trained classification head are included.

The conversion follows timm/deepseek_vit_412m.deepseek_v4_1_flash and the official timm checkpoint mapping. The weights in this repository come from the 1017 release, pinned to revision 1bae0e1088993b9b82de08f59de5be5a34e42170.

Model details

• Backbone: 32 layers, hidden size 1024, 16 attention heads, MLP intermediate size 2816.
• Patch size: 14 × 14; SwiGLU MLP, RMSNorm and axial 2D RoPE.
• Backbone parameters: 411,842,560; stored in BF16.
• Default input: 546 × 546 RGB. Rectangular inputs with dimensions divisible by 14 are supported.
• forward_features() returns final-normalized NHWC patch features.
• forward() returns average-pooled, affine-free RMS-normalized 1024-dimensional embeddings.
• The original aligner groups 3 × 3 patches and projects to 5120 dimensions. It is retained only in the separately provided original-format checkpoint.

Usage

pip install 'timm==1.0.30' torch torchvision safetensors pillow

import torch
import timm
from PIL import Image

model = timm.create_model('hf-hub:gatilin/deepseek_vit_412m.deepseek_v4_1_flash_1017', pretrained=True).eval()
config = timm.data.resolve_model_data_config(model)
transform = timm.data.create_transform(**config, is_training=False)
x = transform(Image.open('image.jpg').convert('RGB')).unsqueeze(0)

with torch.inference_mode():
    embedding = model(x)                  # [1, 1024]
    features = model.forward_features(x)  # [1, 39, 39, 1024]

The default FP32 model loads the BF16 checkpoint into FP32 parameters. For BF16 inference, explicitly move both model and input to a device that supports the required operations and the same dtype.

Files

• model.safetensors: native timm classifier-wrapper checkpoint, without aligner or classification head (~824 MB).
• config.json: timm Hub configuration.
• vision_and_aligner.safetensors: original vision.* and aligner.* keys (~971 MB); this is not a second timm checkpoint.
• vision.py, image_processor.py, source_config.json, LICENSE: original source files from the pinned upstream revision.
• extract_vit.py: reproducible extraction with validated HTTP range requests; downloads only visual tensors from source shard 1.
• validate_vit.py: strict-loading and numerical-equivalence check against the original ViT.
• extraction.json: provenance, parameter count and SHA256 checksums.
• validation.json: validation results and library versions.

To load original-format weights, instantiate ViT and Aligner from vision.py using the source configuration; load the corresponding keys after removing vision. or aligner.. These modules consume patch tensors and patch-grid dimensions rather than raw image batches.

Conversion and validation

The conversion renames attention/MLP keys, replaces the vision. prefix with encoder., and reshapes the linear patch projection from [1024, 588] to [1024, 3, 14, 14]. Tensor data bytes and BF16 precision are preserved.

Strict key/shape loading passed with timm 1.0.30 and torch 2.14.1. FP32 backbone comparisons against the original implementation gave maximum absolute errors of 3.91e-8 for a 28 × 28 input and 2.89e-8 for a 28 × 42 input. These are small-input conversion checks, not downstream accuracy benchmarks.

Preprocessing and limitations

RGB normalization uses mean and standard deviation (0.5, 0.5, 0.5). The timm evaluation transform uses bicubic resize, border padding, and a fixed canvas. The upstream processor uses an adaptive canvas with gray 127 padding; timm uses gray 128. Use the included upstream processor when reproducing the original preprocessing policy.

This is an image feature model. A newly added classification head requires training. The independent backbone does not provide the language-model capabilities of the source multimodal model.

中文说明

本仓库从 DeepSeek-V4.1-Flash-1017 的固定版本重新提取视觉权重，提供可直接通过 timm 加载的独立 ViT，以及保留原始键名的 ViT + Aligner 权重。没有额外训练，没有语言模型权重。主文件为 BF16、411.84M 参数；转换仅涉及键名映射和 patch projection 形状重排。已通过严格加载与两种小尺寸输入的数值一致性验证。默认 timm 预处理与上游动态画布策略存在差异，详见上文。

License

MIT, following the included upstream LICENSE. This is an independently extracted checkpoint; no affiliation with or endorsement by DeepSeek-AI or the timm maintainers is implied.