MiniCPM-OCR-1.0-PL-otoro-si (Q8_0 GGUF)

Modified by otoro si. This is NOT an official OpenBMB release. A community fine-tune of openbmb/MiniCPM-OCR-1.0 (Apache-2.0), trained to stop dropping Polish diacritics.

PL (skrót): MiniCPM-OCR 1.0 gubił polskie ogonki (ł→l, ą→a, ę→e, Ź→Z…). Douczyliśmy go metodą LoRA wyłącznie na danych syntetycznych (bez żadnych prywatnych dokumentów). Odsetek zachowanych polskich liter wzrósł z ok. 69% do 94% na obrazach testowych i z ok. 82% do 96% na prawdziwych skanach. Model nie jest lepszy od Tesseracta ani PP-OCRv6 na czytelnym druku i nie nadaje się do pisma odręcznego (szczegóły niżej).

What this is

Files:

• MiniCPM-OCR-1.0-PL-otoro-si-Q8_0.gguf - the fine-tuned language model
• mmproj-MiniCPM-OCR-1.0-BF16.gguf - vision projector, byte-identical to the one shipped by OpenBMB (sha256 ceed8226…4444682)
• Modelfile - Ollama recipe (two FROM lines + TEMPLATE {{ .Prompt }})

The embedded chat template is the one from the official GGUF (thinking mode disabled). A template taken from the Hugging Face repo makes Ollama put the answer in the thinking field, so do not replace it.

Usage (Ollama, tested)

# put the three files in one folder, then:
ollama create minicpm-ocr-pl -f Modelfile

import base64, requests
img = base64.b64encode(open("page.jpg", "rb").read()).decode()
r = requests.post("http://localhost:11434/api/chat", json={
    "model": "minicpm-ocr-pl", "stream": False,
    "messages": [{"role": "user", "content": "Text Recognition:", "images": [img]}],
    "options": {"temperature": 0, "num_ctx": 8192, "num_predict": 1500}})
print(r.json()["message"]["content"])

Feed the model one paragraph or text block at a time (crops of about 0.6 MP worked well). It was fine-tuned on crops up to ~0.22 MP and evaluated on crops up to ~0.6 MP; whole pages are not tested. The official pipeline of the base model uses a layout detector (PP-DocLayoutV3) plus per-region recognition. llama.cpp directly was not tested by us.

Speed on our machine (RTX 5060 Laptop 8 GB): about 3.3 s per paragraph on GPU, about 15 s per 0.6 MP crop on CPU only (16-core Ryzen 9). Memory in Ollama about 1 GB (LLM) plus the projector.

Results

All numbers below are ours, produced with the same scoring script for every engine. CER = character error rate after whitespace normalisation (lower is better). PL letters = share of Polish diacritic letters (ąćęłńóśźż and capitals) of the reference that appear correctly in the output (higher is better). Word err. = reference words missing from the output.

1. Held-out synthetic images (84 text images, fonts not used in training; tables excluded)

The set includes clean renders, scan-like and photo-like degradations, all-caps lines and decorative fonts. Tesseract fails mostly on the photo-like images (CER 37% without any preprocessing); with proper preprocessing it would do much better.

2. Real scans (11 crops of printed Polish legal and administrative documents, 1,098 words)

On clean printed text, Tesseract and PP-OCRv6 are at least as good as this model, and much faster and lighter. A re-run of this model on another day gave 0.62% CER (Ollama output varies slightly between sessions; one crop changed). The reference transcriptions were typed by hand while also looking at this model's draft output, which may slightly favour it. Eleven crops from four documents is a small sample.

3. Regression check (33 images: English, Chinese, German, French, Czech)

CER of the base model 0.21%, after fine-tuning 0.18% (BF16, transformers pipeline). Table recognition (OTSL) was unchanged on our 10 synthetic tables in BF16/Q8_0.

4. What it cannot do: handwriting and blurry archival scans

Tested on six pages of a blurred microfilm scan with handwriting: only 8 of 339 reference words were found (about 2%), and on two pages the model hallucinated fluent but non-existent English/Latin-looking text. Do not use this model for handwriting or very poor scans, and verify numbers, dates and names against the original in any important document.

Training summary

• Method: LoRA, rank 16, alpha 32, dropout 0.05, all linear layers of the language model; ViT and projector frozen. LR 1e-4 (cosine, 5% warmup), weight decay 0.1, effective batch 16, max length 2048, BF16, 2 epochs. Framework: ms-swift (the fork used in the OpenBMB tutorial).
• Data (all synthetic, no private documents): Polish texts written by Bielik-11B-v3.0-Instruct (SpeakLeash, Apache-2.0) running locally, rendered with a wide set of system fonts (training and test fonts are disjoint families), with scan/photo/low-resolution degradations, justified text, all-caps lines, word lists for rare letters and endings. Round 1: 1,284 samples; round 2 (this model): about 1.8k samples (410 texts including 120 legal-style texts).
• Export: LoRA merged into BF16 weights, converted to GGUF with llama.cpp, quantised to Q8_0. We verified that the published GGUF differs from our internal export only in the general.name and general.description metadata (all 320 tensors identical).
• Training data and the rendering scripts are not included in this repository.

Known weaknesses

• Weakest letters: ł and Ą (about 86-87% kept on the synthetic set; fewer errors on real scans), word-final ć occasionally.
• It sometimes inserts CJK characters in Polish text (the base model does too) and can replace a rare word by a more common one ("hallucinated correction").
• Q8_0 is recommended; a Q4_K_M build of the same fine-tune was slightly weaker and in 2 of 10 synthetic tables dropped the title row.
• Not tested: formulas, charts, seals, whole pages with a layout detector.

Attribution and license

• Base model: MiniCPM-OCR 1.0 by OpenBMB, released under Apache-2.0. The vision projector file in this repository is their unmodified file.
• Synthetic training texts generated with Bielik-11B-v3.0-Instruct by SpeakLeash / ACK Cyfronet AGH (Apache-2.0). No Bielik weights or texts are redistributed here.
• This repository: Apache-2.0 (see LICENSE). The weights are a derivative of the base model; changes are described above. This is a community modification and is not endorsed by or affiliated with OpenBMB.
• Provided "as is", without warranty. OCR output can contain errors and invented text; verify important content.