Xiaomi-OCR-0 — LiteRT-LM (OCR)

On a Galaxy S26 this file reads a paragraph of text in about 10 s and a page in about 13 s on the CPU, engine creation included (2026-11-03). This repository is a conversion of xiaomi-research/Xiaomi-OCR-0 to LiteRT-LM (.litertlm), made with hf-to-litertlm for Google's LiteRT-LM 0.19.0.

Xiaomi-OCR-0 (Xiaomi, 2026-09-30) is a 0.8B OCR model trained from Qwen3.5-0.8B-Base: a 12-layer vision encoder and a Qwen3.5 hybrid text decoder (18 GatedDeltaNet + 6 attention layers). It turns a document page into Markdown (tables as OTSL, formulas as LaTeX), reads text, table and formula regions, and extracts named fields as JSON. This file reads one picture of 768 × 1344 pixels (portrait) per request, which becomes 1,008 image tokens.

Which backend.

• Use the CPU on Android. On both Android phones a CPU request was four to six times as fast as a GPU request (medians; Galaxy S26: 10–13 s against 42–62 s; Pixel 11 Pro: 20–26 s against 92–155 s with the GPU's cache off). The GPU spends most of that time creating its engine (see Known runtime issues).
• Use the GPU (Metal) on the iPhone: on an iPhone 18 Pro a GPU request took 20–22 s. Our one CPU try ended when iOS stopped the app during engine creation (see Known runtime issues).
• On the Mac the GPU passes all 14 of our region tests and the CPU 11 (see Correctness).
• Our one handwritten test page (a dense derivation) fails on the Mac, the Pixel 11 Pro and the iPhone 18 Pro, and the original model misses it at this picture size too.

How to use

ocr_xiaomi.py pads your picture and runs the file with litert-lm:

pip install litert-lm pillow huggingface_hub
hf download litert-community/Xiaomi-OCR-0 --include "Xiaomi-OCR-0_int8.litertlm" --include "samples/*" --local-dir xocr
curl -LO https://raw.githubusercontent.com/john-rocky/hf-to-litertlm/main/xiaomiocr_work/ocr_xiaomi.py
python ocr_xiaomi.py xocr/Xiaomi-OCR-0_int8.litertlm xocr/samples/page_en.png
python ocr_xiaomi.py xocr/Xiaomi-OCR-0_int8.litertlm xocr/samples/table.png --task table
python ocr_xiaomi.py xocr/Xiaomi-OCR-0_int8.litertlm xocr/samples/receipt.png --task kie --keys store_name,date,total

Why the padding: LiteRT-LM resizes every picture to 768 × 1344 without keeping its aspect ratio and without reading the EXIF orientation. So the script applies the orientation, centres the picture on a white canvas of that tall 4:7 shape, and resizes it to 768 × 1344.

--task picks one of the model's five prompts (document is the default; text, table, formula, kie), --prompt sends your own, --backend gpu puts the decoder and the vision encoder on the GPU, and --cache-dir keeps the runtime's caches away from the model file. A picture and a prompt take about 1,030–1,100 of the 4,096 tokens the file holds, so an answer can be up to about 3,000 tokens long.

From your own code, pad the picture the same way (pad_to in the script), send it with the prompt in one user message, and create one Engine per picture (see Known runtime issues).

receipt_to_json in google-ai-edge/litert-samples is an Android app that does this with the LiteRT-LM Kotlin API (litertlm-android 0.19.0): it pads the photo like pad_to, makes one Engine per read, and returns a receipt's store, date, tax and total as JSON from Xiaomi-OCR-0_int8.litertlm on the CPU (the keys can be changed in the app).

The prompts, from the model card ("Task prompts"):

Correctness

The bar: for each test picture, the answer must match what the fp32 original model writes from the same picture at its own size (transformers, greedy). Match means: character error rate ≤ 0.02 for text (≤ 0.05 for photos and pages), at least 18 of 20 table cells, the same LaTeX, and every JSON field. There are 18 test pictures, each sent with one of the model card's prompts. 14 are regions: a text line, a title, three paragraphs (plain, small, photographed), Chinese text, a table, two formulas, a drawn receipt, a receipt photo, and three from the model card (its receipt photo as a document and as fields, its calligraphy sample as a document). 4 are pages: a drawn page, the model card's dense-equation page and handwritten derivation, and a form photo.

LiteRT-LM 0.19.0 everywhere (the Python API on the Mac, the PyPI Android wheel on the Android phones, the iOS framework of the 0.19.0 release on the iPhone), one process per picture. The CPU rows were run with a copy of this file whose only difference is the fp16 declaration on the vision encoder, which LiteRT-LM's CPU path does not read. On the Galaxy S26 we timed the CPU on two pictures (a paragraph and a page) and it wrote the Mac CPU's text for both; we did not run the other 16 there. On the iPhone 18 Pro we ran 13 of the 18 pictures before the phone's battery reached our 20 % stop.

• The handwritten derivation fails on the Mac, the Pixel 11 Pro and the iPhone 18 Pro (we did not run it on the Galaxy S26). At its own size the original model reads it from 8,160 image tokens; this file gets 1,008. At 768 × 1344 the original model misses it too (character error rate 0.25 against its full-size reading).
• The calligraphy sample is a near tie. At its sixth character the original model, given the runtime's 1-D positions, picks 貟 over the simplified 员 by 0.46 against 0.44 in probability. The CPU (Mac and Pixel) and the Pixel GPU take 员 and write the rest of the line in simplified characters too (骑, 兴, 韵). The Mac, Galaxy S26 and iPhone 18 Pro GPUs take 貟 and write the rest of the line like the original.
• Two more near ties. The model card's receipt read as fields keeps the printed label in the GST ID on the Mac CPU and the iPhone 18 Pro GPU (GST ID No :001092886528; the original's top two tokens there are 0.60 against 0.35). The same receipt read as a document gets an extra separator line in 2 of 4 processes on the Mac CPU (0.42 against 0.38). The Pixel CPU reads both like the original.
• Why one fixed picture. The runtime gives the image tokens 1-D positions, and the original model takes each picture at its own size with 3-D positions. Of the fixed sizes we tried on all 14 regions, 768 × 1344 is the only one at which the original model reads all 14 like at full size (with the runtime's 1-D positions; 13 with its own 3-D positions).

Performance

One request = a fresh process with a new Engine, timed inside the process from engine creation to the last token: engine creation, vision encode, prefill and decode. "Paragraph" = a drawn English paragraph (1,029 tokens in, 100 out). "Page" = a drawn English page (1,080 tokens in, 211 out). Three processes per cell, fastest first; (cold) = the runtime's cache directory was empty. Peak = the process's peak resident memory (VmHWM). Bold = the fastest form on that device.

The CPU rows were run with the copy of this file described under Correctness (the same bytes; the CPU path does not read the vision encoder's GPU declaration). On the iPhone, one request is one launch of a test app, and Peak is the app's lifetime peak phys_footprint as iOS reports it, not the VmHWM of the Android rows.

On a Mac M4 Max (LiteRT-LM 0.19.0 Python API, 4 threads) the CPU path took 7.1 s (cold) and 8.2 s for the paragraph and 8.6 and 13.5 s for the page, timed as the whole process.

On both Android phones the GPU prefills faster than the CPU, but it decodes slower and spends most of each request creating its engine (37–53 s on the Galaxy S26, 72–108 s on the Pixel 11 Pro; see Known runtime issues). The Pixel 11 Pro's GPU decoded at 4–8 tok/s.

Conversion notes

• Source: xiaomi-research/Xiaomi-OCR-0 at revision bbd8cf8d (873,438,784 parameters in the checkpoint, of which the 20,452,864 of the mtp.* layer are not used for generation).
• Text decoder: 7 signatures (prefill 1024, 256, 64, 16, 4, 1 and decode), a 4,096-token KV budget, the embedder outside the decoder for the fast_vlm path. Dynamic int8 on the linear layers and the embedding; the 36 GatedDeltaNet decay/gate projections (in_proj_a / in_proj_b) stay float; fp32 activations declared for the GPU. Toolchain: a litert-torch checkout at commit 115a136 (version string 0.10.0) plus the Qwen3.5 hybrid patch of hf-to-litertlm, litert-converter 0.3.0, ai-edge-quantizer 0.8.0, ai-edge-litert 2.1.6, transformers 5.14.1, torch 2.12.1.
• Vision: the 12-layer encoder and the 2×2 merger at a fixed 768 × 1344 picture (84 rows of 48 patches), its LayerNorm inputs scaled by powers of two so fp16 holds, exported with litert-torch 0.9.3 / litert-converter 0.4.0; encoder fp16, adapter int8 (ai-edge-quantizer 0.9.0); fp16 activations declared for the GPU vision executor.
• Template: a ChatML template that renders, for one turn, the same text as the model's own template with thinking off (its default): an empty <think>\n\n</think>\n\n block before every answer, stops <|im_end|> and <|endoftext|>, no start token. The tokenizer is the repository's own tokenizer.json.
• Packing: litert-lm-builder 0.15.0, plus the ExecutorMetadata section the hybrid runtime binds its 48 state buffers with (packed with litert-lm 0.18.1).
• Reproduction: REPRODUCE.md · machine-readable facts: litertlm_manifest.json.

Known runtime issues

• One Engine per picture. This hybrid decoder carries its recurrent state into the next conversation on the same Engine (LiteRT-LM #3165), so every request in the tables above creates its own Engine. Our check: 6 pictures one after another on one Engine (LiteRT-LM 0.19.0, Mac, the CPU rows' copy of this file, which on the GPU runs the vision encoder in fp32). The fifth answer, a formula, differed from its fresh-process answer on the CPU and on the GPU; the other five were the same. On the GPU each conversation took 1.8–5.7 s once the Engine existed; creating it took 56 s. A fix landed on LiteRT-LM main on 2026-11-03 (e1f86b05, the internal CL that continued PR #3945). No release carries it yet (v0.19.1 does not); this note will be updated when one does.
• The GPU builds most of its programs again every time. On the Galaxy S26, at each later engine creation 5 of the 8 GPU programs in the runtime's cache fail to load with DATA_LOSS (LiteRT-LM #4178) and are compiled again, and the cache files next to the model grow at each creation (254 MB after the first, 1,412 MB after six) until you delete them. Engine creation took 37–53 s there with the cache and 37–70 s without it; on the Pixel 11 Pro we ran the GPU with the cache off only, and its engine creation took 72–108 s over 15 processes. The iPhone 18 Pro (Metal) did the same: at each later engine creation 5 of the 8 GPU programs failed with DATA_LOSS and were compiled again. Its cache directory grew by about 494 MB per creation (1,309 MB after the first, 3,283 MB after the fifth). Engine creation took 16.9–17.1 s with a populated cache, 17.7 s with an empty one, and 17.2–18.5 s with the cache off (13 processes).
• Later turns. From the second turn of a conversation on, LiteRT-LM puts the previous answer's stop token into the KV cache with a zero embedding (this file holds the embedding table in its own section and the decoder takes embeddings). So later answers in a conversation can differ from the answer the model gives to the same messages in one prompt (read in the source of LiteRT-LM 0.16.0 to 0.18.1; measured with 0.18.1 on WeVisOne-VL-2B and Qwen2.5-Coder-1.5B-Instruct, not on this file). One question per conversation avoids it, as does a new conversation that starts with the earlier messages and the new question.
• 8 GB Android phones. We have not run this file on an 8 GB Android phone. Use the CPU backend on 8 GB Android phones and keep the runtime's cache directory: with it, the CPU process peaked at 3.3 GiB on the Pixel 11 Pro and 4.2 GiB on the Galaxy S26 (up to 6.7 GiB in our cache-off runs). The GPU runs the decoder with fp32 activations; on the Galaxy S26 its process peaked at 3.8 GiB, plus up to 4.3 GiB of GPU allocations.
• Where the GPU was run. The Mac (Metal), the Galaxy S26 (Adreno), the Pixel 11 Pro (PowerVR) and the iPhone 18 Pro (Metal). There was no Mali phone, so an Arm Mali GPU is not verified with this file.
• iPhone CPU. Our one CPU try on the iPhone 18 Pro ended 3.9 s after launch: iOS stopped the app (signal 9) while it was creating the CPU engine. The app used the iOS framework of the LiteRT-LM 0.19.0 release and had the increased-memory-limit entitlement. We could not read the crash report, so the cause is not known. The GPU (Metal) ran all 13 pictures we gave it; its peak footprint was up to 5.03 GiB in those runs (cache off) and 4.3–4.7 GiB in the timed runs (cache on).
• Mac GPU, long outputs: not verified on this file. On the Mac GPU, LiteRT-LM 0.19.0's default sampler path made the greedy answers of a Qwen3.5-4B finetune differ from run to run and broke its long code answers; we have not checked this file run to run. Each picture ran once per backend, and the longest answer, the dense-equation page (4,488 characters), passed the bar on the Mac GPU.
• Tool calls are not parsed. The template renders no tool definitions, and this file's metadata carries no tool-call config.

License and changes

Apache-2.0, inherited from xiaomi-research/Xiaomi-OCR-0. Changes from the original work: exported to the LiteRT-LM format at a fixed 768 × 1344 picture; the text decoder's linear layers and the embedding quantized to int8; the vision encoder cast to fp16 and its adapter quantized to int8; the chat template with thinking off; stop tokens <|im_end|> and <|endoftext|>. The sample pictures in samples/ were drawn with the fixture script of hf-to-litertlm (DejaVu fonts). Problems with these files: open a discussion in this repo.