LightOnOCR-3-1B-GGUF
LightOnOCR-3-1B, developed by lightonai, is a highly performant, lightweight OCR and end-to-end document understanding model designed as a seamless, drop-in upgrade for existing LightOnOCR-2 deployments. Released under the Apache 2.0 license, it retains the proven LightOnOCR-2-1B architecture while introducing powerful new visual understanding capabilities, including a grounding mode (triggered by the
groundingprompt) that outputs labeled bounding boxes for all document elements in normalized page coordinates, short descriptions for images, and structured HTML data extraction from charts. Alongside its default transcription mode that outputs full-page markdown text from an empty prompt, the model is highly versatile—effortlessly handling complex multi-column layouts, tables, receipts, forms, handwriting, and math notation—offering a ready-to-use, single-model alternative to traditional, multi-stage document processing pipelines when images are optimally preprocessed at 200 DPI.
Model Files
| File Name | Quant Type | File Size | File Link | Description |
|---|---|---|---|---|
| LightOnOCR-3-1B.BF16.gguf | BF16 | 1.2 GB | Link | Full BF16 weights. Highest quality, largest file size. |
| LightOnOCR-3-1B.Q3_K_L.gguf | Q3_K_L | 368 MB | Link | Lower quality but usable, good for low RAM availability. |
| LightOnOCR-3-1B.Q3_K_M.gguf | Q3_K_M | 347 MB | Link | Low quality. |
| LightOnOCR-3-1B.Q4_K_M.gguf | Q4_K_M | 397 MB | Link | Good quality, default size for most use cases, recommended. |
| LightOnOCR-3-1B.Q4_K_S.gguf | Q4_K_S | 383 MB | Link | Slightly lower quality with more space savings, recommended. |
| LightOnOCR-3-1B.Q5_K_M.gguf | Q5_K_M | 444 MB | Link | High quality, recommended. |
| LightOnOCR-3-1B.Q5_K_S.gguf | Q5_K_S | 437 MB | Link | High quality, recommended. |
| LightOnOCR-3-1B.Q6_K.gguf | Q6_K | 495 MB | Link | Very high quality, near perfect, recommended. |
| LightOnOCR-3-1B.Q8_0.gguf | Q8_0 | 639 MB | Link | Extremely high quality, generally unneeded but max available quant. |
| LightOnOCR-3-1B.mmproj-bf16.gguf | mmproj-bf16 | 829 MB | Link | Multimodal projection file in BF16 format. Used for vision/language models. |
| LightOnOCR-3-1B.mmproj-q8_0.gguf | mmproj-q8_0 | 449 MB | Link | Multimodal projection file in Q8_0 quantization. Smaller size for vision capabilities. |
llama.cpp
LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp