LightOnOCR-3-0.8B — MLX 8-bit (flat)
The 8-bit quantized MLX conversion of
LightOnOCR-3-0.8B — LightOn's
lightest end-to-end OCR / document-understanding model (Qwen3.5 vision-language
architecture, ~0.9 GB here; language model 8-bit affine, group 64, vision tower
bfloat16) — taken from the 8bit/ subfolder of
prithivMLmods/LightOnOCR-3-0.8B-MLX
and flattened into a single-model repo. The weights are bit-identical to that
subfolder; nothing was re-quantized.
Why this flat copy exists
prithivMLmods/LightOnOCR-3-0.8B-MLX keeps its weight variants in subfolders
(root = bf16, 4bit/, 8bit/), and mlx-vlm's OpenAI-compatible server has no
--subfolder flag — it can only load a flat model directory. This repo removes the
workaround of materializing the subfolder locally before every server start.
How this repo was made
On 2026-11-15 (with mlx-vlm 0.7.12 present for the smoke test), the subfolder files were downloaded from the source repo and uploaded unchanged:
hf_hub_download("prithivMLmods/LightOnOCR-3-0.8B-MLX", "<file>", subfolder="8bit")
plus a flattened layout and this README. For the 4-bit or bf16 variants, see the source repo's subfolders.
Use
Requires mlx-vlm (tested with 0.7.12):
pip install -U mlx-vlm
from mlx_vlm import load, generate
model, processor = load("fwgpiyawudk/LightOnOCR-3-0.8B-MLX-8bit")
# plain transcription: the image is the whole prompt, no text item.
# grounding mode: pass prompt="grounding" instead.
result = generate(model, processor, prompt="", image="page.png",
max_tokens=4096, temperature=0.2, top_p=0.9)
print(result.text)
OpenAI-compatible server (the image content part goes first, then the text item — none at all for plain mode):
python -m mlx_vlm.server --model fwgpiyawudk/LightOnOCR-3-0.8B-MLX-8bit --port 8080
Sampling per the official model card's vLLM example:
temperature 0.2, top_p 0.9, max_tokens 4096 (the README tip prefers 0.1 — anything
but greedy). The model is trained on exactly two instructions: plain transcription
(send the image alone, no text item) and grounding (image first, then the text
"grounding"; one  text block per line, 0-1000 boxes).
Preprocessing: RGB, downscale so the longest edge is <= 2048 px (never upscale).
Thinking is disabled (the chat template default — never start the server with
--enable-thinking / MLX_VLM_ENABLE_THINKING=1).
The full usage documentation, benchmarks and the official client live in the source model card.
Licence
Apache-2.0, the licence of the model and of the source MLX conversion.