This repository packages the vision Tower and merger from MiMo V2.6 Flash.
| File | Tensors | What it holds |
|---|---|---|
model.safetensors |
364 | Tower blocks, patch embedding and merger, from the parent's visual.* |
modeling_mimo_vision.py |
Vision classes extracted from the parent, loaded with trust_remote_code |
|
config.json, preprocessor_config.json |
Vision configuration and the parent's Qwen2VLImageProcessor settings |
|
export_manifest.json |
Source pin and tensor count | |
LICENSE, LICENSE-CODE, NOTICE |
Weight and code licenses |
| Component | Details |
|---|---|
| Tower | 28 blocks, 1280 hidden, SwiGLU 4608, 32 query heads, 8 KV heads, head dimension 64, RMSNorm, 2D RoPE |
| Patches | 2 frames x 16 x 16 pixels; a still image is repeated as two frames |
| Attention | Full attention at blocks 0, 9, 18 and 27; the other blocks use a ±64-token window and a per-head sink bias on key column 0 |
| Merger | LayerNorm(1280), 2x2 concatenation to 5120, Linear(5120, 5120), GELU, Linear(5120, 4096), no biases |
At 448 the bench reports 784 Tower tokens of width 1280 (restored to raster
order), 196 merged tokens of width 5120, and 196 projected tokens of width 4096.
See examples/inference.py for image feature extraction.
AutoModel.from_pretrained(..., trust_remote_code=True) loads modeling_mimo_vision.py.
The parity check compares every exported tensor with the pinned parent using torch.equal, then checks all stages against the parent's own MiMoVisionTransformer.
The export script reads the visual.* tensors from the pinned parent and writes the original BF16 tensors.
Xiaomi released the MiMo V2.6 Flash weights and the vision implementation.
Weights are MIT, as declared on the source model card. modeling_mimo_vision.py keeps its Apache-2.0 header; see LICENSE-CODE and NOTICE.
XiaomiMiMo/MiMo-V2.6-Flash-RL