LightOnOCR-3-0.8B GGUF
GGUF conversion of lightonai/LightOnOCR-3-0.8B, made for local inference with llama.cpp. The original model is licensed under Apache 2.0.
| File | Size | Purpose |
|---|---|---|
model-Q4_K_M.gguf |
542 MB | Default quantized language model with MTP |
mmproj-F16.gguf |
205 MB | Vision encoder and projector, required for images |
llama.cpp 0.6.0 or newer can download the vision file automatically and enable MTP decoding:
llama-server -hf staghado/LightOnOCR-3-0.8B-GGUF:Q4_K_M \
--spec-type draft-mtp \
--chat-template-kwargs '{"enable_thinking":false}'
Send an image with an empty text prompt for transcription. The model also supports the grounding prompt for boxes and labels, as described in the original model card.
The Q4_K_M model and matching projector were smoke tested with llama.cpp on the receipt image linked from the original model card. This is a functionality check, not a quality benchmark.