LightOnOCR-3-1B (GGUF)

GGUF quantizations of lightonai/LightOnOCR-3-1B — the LightOnOCR-3 model that keeps the LightOnOCR-2-1B architecture, so switching requires no change, and adds grounding, image description and chart extraction — optimized for local inference with llama.cpp and Ollama, with its vision projector. All quants use an importance matrix.

Quick Start

With Ollama

ollama run richardyoung/lightonocr-3-1b
# or straight from Hugging Face
ollama run hf.co/richardyoung/LightOnOCR-3-1B-GGUF:Q8_0

With llama.cpp

huggingface-cli download richardyoung/LightOnOCR-3-1B-GGUF \
    --include "*Q8_0*" "mmproj*" --local-dir ./models

./llama-mtmd-cli -m ./models/LightOnOCR-3-1B-Q8_0.gguf \
    --mmproj ./models/mmproj-LightOnOCR-3-1B-F16.gguf \
    --image page.png -p "grounding"

Prompts: send the image with an empty prompt for plain transcription (Markdown), or with the prompt grounding to get every block with a label and a 0–1000 bounding box, image descriptions and chart data as tables. LightOn notes that other instructions are out of distribution. Recommended sampling: temperature 0.2, top_p 0.9 (set in the Ollama tags).

Why This Model?

Drop-in upgrade for LightOnOCR-2-1B deployments
Two modes: plain transcription, or grounding with labels and bounding boxes
Handles tables, receipts, forms, charts, multi-column layouts, handwriting and math
Apache-2.0: research and commercial use

Use Cases

Upgrading existing LightOnOCR-2 pipelines
Handwriting and form OCR
Layout-aware extraction

Available Quantizations

File	Size	Use Case
`LightOnOCR-3-1B-Q4_K_M.gguf`	0.4 GB	Smallest
`LightOnOCR-3-1B-Q5_K_M.gguf`	0.44 GB	Higher quality
`LightOnOCR-3-1B-Q6_K.gguf`	0.5 GB	Very high quality
`LightOnOCR-3-1B-Q8_0.gguf`	0.64 GB	Recommended — best accuracy for a small OCR model
`mmproj-LightOnOCR-3-1B-F16.gguf`	0.82 GB	Vision projector (required for images)
`LightOnOCR-3-1B-imatrix.gguf`	–	Importance matrix used for the quants (calibration_datav3)

Verified

Tested through Ollama before upload (Q4_K_M and Q8_0) on a synthetic invoice with an empty prompt; Q4_K_M returned: 'INVOICE 4471 / Date: 2026-09-28 / Widgets x3 ....... $42.00 / TOTAL: $86.20'.

Also Available

LightOnOCR-3-4B — most accurate
LightOnOCR-3-0.8B — fastest

Other Models by richardyoung

New Heretic models (reproducible, listed in the Heretic Grimoire): LFM2.5-1.2B | Granite-4.2-3B | Granite-4.2-8B | Olmo-3-7B-Think | Phi-4-mini | DeepSeek-R1-0528-Qwen3-8B | Agents-A1-4B | FrogNano-4B | Muse-Glimmer-30B | Hemmingway-1
Abliterated/Uncensored models: Qwen2.5-7B | Qwen3-14B | DeepSeek-R1-32B | Qwen3-8B
OCR & Vision: MiniCPM-V-4.6 GGUF | dots.mocr GGUF | Unlimited-OCR GGUF | olmOCR GGUF | LightOnOCR-3-4B GGUF | WeVisDoc-4B GGUF | Xiaomi-OCR-0 GGUF
MLX quantizations (Apple Silicon): Kimi-K2 series | olmOCR MLX
Healthcare/Medical: Synthea 575K patients dataset | CardioEmbed
Research: LLM Instruction-Following Evaluation (arxiv:2510.18892)

Built & maintained by Richard Young · DeepNeuro

richardyoung/LightOnOCR-3-1B-GGUF