MiniCPM-V-4.7-1B — LiteRT-LM (image + text)

openbmb/MiniCPM-V-4.7-1B converted to the LiteRT-LM (.litertlm) format for Google's LiteRT-LM runtime, as an image + text bundle on its fast_vlm path. Tested on litert-lm 0.18.0 on a Mac M4 Max and on a Galaxy S26, a Pixel 8a and an iPhone 18 Pro: on the Galaxy S26 the image bundle returns the first token of an answer about a 448×448 photo 0.93 s after the request on the GPU, and decodes text at 36.5 tokens/s on the CPU (2026-10-13). Reproduction: REPRODUCE.md · machine-readable facts: litertlm_manifest.json.

MiniCPM-V 4.7 (OpenBMB, released 2026-10-12) is a 1B vision-language model: a Qwen3.5-style hybrid text decoder (18 GatedDeltaNet linear-attention layers + 6 full-attention layers, hidden 1024, vocab 248,094) under a 27-layer SigLIP-style vision encoder that compresses visual tokens early, inside the encoder (a 2×2 window-attention merger after layer 6, then a 2×2 merger). OpenBMB positions it for single-image, multi-image and video understanding on phones. This bundle carries the single-image path in the model's "Instruct" mode (direct answers, thinking off).

Which file: MiniCPM-V-4.7-1B-VL_int8.litertlm on macOS, iPhone and 12 GB-class Android phones (the one measured on the Galaxy S26 and the iPhone 18 Pro); MiniCPM-V-4.7-1B-VL_int8vis.litertlm on the Pixel 8a (measured there on the CPU); MiniCPM-V-4.7-1B_int8.litertlm when no image is involved. All three share the decoder bytes, so the text behaviour is the same.

What the image path does

LiteRT-LM's fast_vlm contract takes one image per message, resizes it to a fixed 448×448, runs the bundle's vision encoder (27 layers, window merger) and adapter, and splices 64 image tokens into the prompt where the chat template puts <image_id>0</image_id><image>…</image> — the same layout, token ids and count the model's own processor produces for an image whose area is at most 448×448 (its single-thumbnail path). Two things differ from the Transformers pipeline and were measured on the fp32 model before conversion (three fixtures × three questions, 64 new tokens each):

• Larger images are resized to the 448×448 square instead of being tiled into a thumbnail plus up to nine slices. Against the tiled run the answers were paraphrases of the same content in 9 of 9 cases (no case changed what the image contains, none was identical).
• The runtime feeds plain 1-D positions, so the model's canvas M-RoPE collapses to ordinary RoPE on image tokens. Against the canvas-position run: paraphrases in 9 of 9 cases, none identical, none wrong.

The bundle therefore answers questions about a picture (describe, colours, where, what is written), at the model's 16× token compression, one image per conversation. Video, multi-image prompts and the 4× ("finer detail") mode are not in this bundle.

Correctness

• Text decoder: teacher-forced logits of the float export against the fp32 model at 48 positions, top-1 identical at every position, Pearson 1.0000 (max |Δlogit| 5.3e-5); the int8 file 47 of 48 (one near-tie). The derived text checkpoint fed to the exporter reproduces the full model's text logits bit for bit.
• Vision: the re-authored tower against the model's get_image_features on four 448×448 inputs, correlation 0.99999999997 (fp32); the fp16 encoder + fp16 adapter end-to-end correlation 1.000000; the int8 encoder of the _int8vis file 0.9954 (worst per-token cosine 0.911).
• 8-question check (Mac M4 Max, litert-lm 0.18.0, --temperature 0, one litert-lm run process per question): CPU 8/8; GPU 8/8 and word-for-word the fp32 model's answers (the CPU differs on one rhyme: "blue" where the fp32 model says "white"). Prompt lengths 12–51 tokens each answered correctly in a fresh process (40/40, CPU). Image: three photos (two cats on a pink couch, two kitchens) × three questions, 9/9 grounded on the CPU and 9/9 on the GPU, vision on the GPU as well.
• On the phones: Pixel 8a _int8vis CPU 3/3 photos grounded; Galaxy S26 CPU and GPU grounded, text answers "Paris." on both; iPhone 18 Pro Metal and CPU answer the eight questions (Metal 8/8, CPU 7/8 — the CPU's "Four days are in a week." is the same text the Mac CPU gives on that combined prompt) and read a text probe image as "Yes … SAFETY, PROBE, TEXT" and a fractal as "No".

Usage

litert-lm run ./MiniCPM-V-4.7-1B-VL_int8.litertlm --prompt "What is the capital of France? Answer in one sentence."
litert-lm run ./MiniCPM-V-4.7-1B-VL_int8.litertlm --prompt "What is in this image?" --attachment photo.png
litert-lm run ./MiniCPM-V-4.7-1B-VL_int8.litertlm --prompt "What is in this image?" --attachment photo.png --backend gpu --cache no
litert-lm benchmark ./MiniCPM-V-4.7-1B-VL_int8.litertlm -p 256 -d 256 --runs 3 --cache no --backend cpu

Without --backend, run uses the CPU; --vision-backend gpu also moves the vision encoder to the GPU. Without --cache no, the GPU run writes a compiled-weights cache next to the model. On Android, litert_lm_advanced_main takes the image inline: --input_prompt='[image:/data/local/tmp/photo.png]What is in this image?' --vision_backend=cpu (see android-gpu.md). On iOS, load the bundle with the vision modality enabled.

Create one Engine per conversation: on this hybrid model a Conversation opened on an Engine that already served another one continues from that conversation's recurrent state (LiteRT-LM #3165); measured on the iPhone 18 Pro, the second and third conversations on one Engine gave different, broken answers to the same eight questions, a fresh Engine the same answers every time.

The chat template is a ChatML template that accepts both the string and the 0.18 content-parts form, renders an image item as <image_id>N</image_id><image><image_soft_token></image> and keeps the model's empty <think>\n\n</think> scaffold (thinking off); stops are <|im_end|> and <|endoftext|>, no start token (the model's bos_token field names <|im_start|>, which its own template already emits). A second turn prefills exactly the tokens the Transformers template adds (the newline after an assistant turn is placed at the start of the next turn, because the runtime keeps the stop token in the cache and prefills only the increment). Tool calls are not parsed (the model's <function=…> XML form is returned as text). The model's own template defaults to thinking; this bundle does not expose a switch — in a probe with a <think> prefill the model closed its thought and answered 9 of 9 times, so a thinking variant is possible as a metadata re-pack.

Performance

Mac M4 Max, litert-lm 0.18.0, benchmark … -p 256 -d 256 --runs 3 --cache no (one warmup, mean of 3); GPU is the runtime's WebGPU backend on Apple silicon. Image rows are one cold run process (load + engine + vision + prefill + answer).

Phones, Android 16, prebuilt litert_lm_advanced_main published 2026-09-18 (sha256 adac974b…), --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3; the image rows are one process with the 448×448 photo and "What is in this image?", first token = vision encode + prefill of the 88-token prompt.

The CPU rows are sustained-run numbers: the phone lowered its CPU clock in 14 of 16 samples taken 2 s apart; the GPU clock limit fell from 1300 MHz in 13 of 44 samples. The GPU init is the cold number of this kit, which rebuilds its program cache on every process. The NPU path of litert-lm 0.18.0 takes static-range int8 and float graphs; this dynamic-range int8 decoder is rejected by the Qualcomm compiler (DynamicUpdateSlice) and the file runs on the CPU and GPU.

The decoder's GPU engine did not build on this 8 GB phone: available memory fell below our 400,000 kB safety line while the fourth of seven signatures was initializing, and the harness stopped the process (one attempt, by rule). Read the Android GPU rows above as Galaxy S26 (Adreno) only: on this 8 GB Mali phone the decoder GPU engine does not build within that memory line, so use the CPU backend on 8 GB phones (the _int8vis file).

Medians of three processes, one Engine per process (the 138-token eight-question prompt; the image probe is a 560×560 text image + question, 117 tokens). With one Engine per conversation the engine load counts once per conversation, so a short text chat completes sooner on the CPU; long answers are faster on Metal.

Conversion notes

• Source and stack: openbmb/MiniCPM-V-4.7-1B at revision 984a89aa. Text decoder: a text-only checkpoint derived from the VLM weights (model.language_model.* → model.*, untied lm_head, qwen3_5_text config; logits bit-identical to the full model) exported with litert-torch at commit 115a136 plus the Qwen3.5 hybrid patch of hf-to-litertlm (qwen35_work/), transformers 5.14.1, ai-edge-quantizer 0.8.0; 7 signatures (prefill 1024, 256, 64, 16, 4, 1 and decode) over a 4096-token KV budget, embedder externalized for the fast_vlm path. Vision: the tower re-authored in plain PyTorch from the checkpoint weights (window attention as a block-diagonal mask in raster order, the two 2×2 merges as strided slices, LayerNorm inputs pre-scaled by calibrated powers of two for fp16), exported with litert-torch 0.9.3 / litert-converter 0.4.0 (no GATHER, FLEX or custom ops), reference = the model's get_image_features in transformers 5.19.0. Packed with litert-lm-builder 0.15.0 plus the ExecutorMetadata section the hybrid runtime binds its 48 state buffers with.
• Quantization: dynamic int8 on the decoder's linears and embedding; the 36 GatedDeltaNet decay/gate projections (in_proj_a / in_proj_b) stay float — on the CPU their run-time int8 activation quantization made the Qwen3.5 siblings loop on arithmetic. The vision adapter stays fp16: its int8 form lost correlation (0.9969 end to end) through the same CPU activation quantization. The GPU runs the decoder with fp32 activations (declared in the bundle; the family's fp16 default breaks on a layer-0 head).
• One graph detail worth knowing: the vocabulary (248,094) is not a multiple of 4, so the GPU sampler's top-k kernel logs an error and works on the first 248,092 logits — the two ids left out are the video <timestamp> tokens, which never appear in image or text answers. Padding the logits to 248,096 removed the message and changed the GPU decode speed by −0.5 % (within run-to-run noise), so the file ships unpadded.
• Template: metadata re-pack after the device runs (the newline placement for multi-turn prompts described under Usage); the TFLite sections, tokenizer and executor metadata are byte-identical to the measured files.

Known runtime issues

From the second turn of a conversation on, LiteRT-LM puts the previous answer's stop token into the KV cache with a zero embedding, because MiniCPM-V-4.7-1B-VL_int8.litertlm and MiniCPM-V-4.7-1B-VL_int8vis.litertlm hold the embedding table in its own section and the decoder takes embeddings, so later answers in a conversation can differ from the answer the model gives to the same messages in one prompt (read in the source of LiteRT-LM 0.16.0 to 0.18.1; measured with 0.18.1 on WeVisOne-VL-2B and Qwen2.5-Coder-1.5B-Instruct, not on these files). MiniCPM-V-4.7-1B_int8.litertlm has no embedder section (the decoder looks the tokens up itself) and is not affected. One question per conversation avoids it, as does a new conversation that starts with the earlier messages and the new question.

GPU engine creation does not get faster after the first. On the iPhone 18 Pro (Metal) and the Galaxy S26 (OpenCL), each process that found a program cache failed to load 5 of the decoder's programs from it (Deserialization failed: DATA_LOSS, LiteRT-LM #4178), so each engine creation builds them again: 16.4–16.9 s on the iPhone 18 Pro, about 50–53 s on the Galaxy S26. Whether the cache file next to the model grows at each creation was not checked on these files; on the Qwen3.5 0.8B, 2B and 4B files it grows by 319 to 952 MB per creation.

iOS GPU apps and the memory limit. The iPhone 18 Pro rows above ran in an app signed with the com.apple.developer.kernel.increased-memory-limit and com.apple.developer.kernel.extended-virtual-addressing entitlements, and their Metal peaks are 3,517–5,998 MB. These files were not run without them. On an iPhone 18 Pro, an app without them had 3.49 GB available at launch and was killed at 3.54 GB while creating a Qwen3.5-0.8B int8 GPU engine.

VL_int8vis with its vision encoder on the GPU. Keep the VL_int8vis file's vision encoder on the CPU: on the Mac GPU (WebGPU, litert-lm 0.18.1, decoder on the CPU) with a cache directory it did not read the test picture, a 448×448 paragraph of text, in any of 4 processes, the empty directory's first process included: it answered The image contains a block of text: "none". each time, and the log of these runs carries Validation error lines. With the cache off (cache_dir=":nocache") the same picture came back as the paragraph, 2 of 2, and litert-lm run --cache no also read it. The rows above that run a vision encoder on a GPU use VL_int8; no phone GPU ran this file's vision encoder.

Mac GPU, long outputs: not verified on these files. On a Qwen3.5-4B build (int8 and Mixed INT4) on an Apple M4 Max, the default GPU path, where the sampler also writes the next decode step's inputs (the log says Sampler will handle decode input tensors.), gave a different greedy reply on each run and emitted tokens the model itself scores at a log probability of −16 to −23 (litert_lm_advanced_main built from LiteRT-LM main at dab2e708); with litert-lm 0.19.0 from PyPI, which always takes this path, 16 of 22 long code replies failed node --check. With --sampler_handles_input=false, or --sampler_backend=cpu, on that C++ CLI the reply was the same on every run and 0 of 7 failed. The Python, Swift, Kotlin and C APIs of 0.19.0 have no setting for this.

License and changes

Apache-2.0, inherited from openbmb/MiniCPM-V-4.7-1B. Changes from the original work: exported to the LiteRT-LM format, text decoder quantized to int8 and the vision encoder to fp16 (or int8 in the _int8vis file) as described above; the chat template is a dual-form ChatML template with thinking off; stop tokens <|im_end|> and <|endoftext|>. Problems with these files: open a discussion in this repo.