openbmb/MiniCPM-OCR-1.0-gguf

Demo (CN / Global) · GitHub · Free API · SDK usage · DSpark · Skill

This repository hosts the GGUF build of MiniCPM-OCR 1.0 for llama.cpp. For the original weights, the full model card, and benchmarks, see the base repository.

MiniCPM-OCR 1.0 (1.3B) parses documents into text, tables, formulas, charts, and seals, and supports key information extraction, for Chinese and English documents and PDFs.

Files

Every tier needs the vision projector as well — --model takes the LLM GGUF, --mmproj takes the projector:

The projector is 4× only; there is no 16× variant for this model.

Which tier

Prefer Q8_0 for code- or XML-heavy pages.

Quick start

Use with the MiniCPM-OCR SDK

Required before using either the CLI or Python API: deploy and start the OCR model service first, and wait until it is ready to accept requests. The SDK examples below are clients of that running service; they do not deploy or start the OCR model service automatically.

Use the official minicpm-ocr SDK to parse local images or PDFs with this GGUF checkpoint. The SDK detects layout with PP-DocLayoutV3, recognises each region, and saves structured JSON and Markdown in reading order.

Install Python 3.10+ and the SDK:

pip install "minicpm-ocr[all]"

Install minicpm-ocr for your platform using the installation guide.

Prerequisite: deploy and start the model service

Before running the SDK, start a llama.cpp recognition service using this repository's MiniCPM-OCR-1_0-Q8_0.gguf and the required mmproj-model-f16-4x.gguf vision projector. Follow the llama.cpp deployment guide. For the SDK examples below, configure the service with --port 8080, --alias minicpm-ocr, --ctx-size 8192, --parallel 1, and --jinja --reasoning off. If you keep the guide's port 18499, change the SDK URL to match. DSpark setup, when used, is covered by the same guide.

The SDK uses the llamacpp backend, an 8192-token context, and one recognition request at a time. Layout detection runs on CPU in these examples; the recognition service can independently use Metal, CUDA, or CPU. If you increase the service context per request slot, update engine.max_model_len accordingly.

CLI

Start the model service described above and wait until it is ready before running this command. Save a local image as demo.png, then run:

minicpmvocr parse demo.png \
  --backend llamacpp \
  --server-url http://127.0.0.1:8080 \
  --model minicpm-ocr \
  --device cpu \
  --set engine.max_model_len=8192 \
  --workers 1 \
  --set pipeline.queue_mode=serial \
  --set pipeline.overlap_layout=false \
  --set run.run_root=output \
  --output output/demo.md

The CLI automatically downloads PP-DocLayoutV3 to models/PP-DocLayoutV3_infer/ when needed. The final Markdown is saved to output/demo.md; structured JSON and per-page Markdown are kept in output/<run_tag>/results_json/ and output/<run_tag>/output/.

Python API

The model service described above must already be running before you execute this Python example. Set engine.server_url and engine.model_name to match that service.

The same pipeline is available from Python:

from importlib.resources import files
from pathlib import Path

from minicpmvocr import load_config, pipelined_run
from minicpmvocr.layouts.download import ensure_layout_model


def main():
    layout_model = ensure_layout_model(Path("models/PP-DocLayoutV3_infer"))

    cfg = load_config(
        str(files("minicpmvocr").joinpath("configs/minicpmvocr.yaml")),
        overrides={
            "input": {"file_path": str(Path("demo.png").resolve())},
            "engine": {
                "type": "llamacpp",
                "server_url": "http://127.0.0.1:8080",
                "model_name": "minicpm-ocr",
                "max_model_len": 8192,
            },
            "layout": {
                "model_dir": str(layout_model),
                "device": "cpu",
            },
            "pipeline": {
                "overlap_layout": False,
                "queue_mode": "serial",
                "max_workers": 1,
            },
            "run": {"run_root": str(Path("output").resolve())},
        },
    )

    pipelined_run(cfg)

    print("JSON directory:", cfg["_meta"]["result_json_dir"])
    print("Markdown directory:", cfg["_meta"]["output_dir"])


if __name__ == "__main__":
    main()

ensure_layout_model() reuses or downloads the official layout model. Results are written to output/<run_tag>/results_json/ and output/<run_tag>/output/. To parse a PDF, replace demo.png with its local path.

For CUDA layout detection, change --device cpu to --device cuda:0 in the CLI, or set layout.device to cuda:0 in Python, after installing the matching PaddlePaddle runtime. Keep SDK recognition concurrency aligned with the llama.cpp service's request slots.

Run minicpmvocr parse --help for CLI options, or see the SDK API reference for configuration and document-level Markdown export.

Performance

Single RTX 5090 (32 GB), F16 weights: model + projector load ≈2.3 s; one table crop (1,312 prompt / 265 output tokens) ≈1.0 s; VRAM ≈26 GB at --ctx-size 40960. CPU-only (48 threads): one table image end-to-end (layout + recognition + assembly) ≈21 s.

Build llama.cpp and further options: llama.cpp deployment guide.

Related checkpoints

Speculative decoding works in llama.cpp with the DSpark draft GGUF via --spec-type draft-dspark.

FAQ

Installation, model downloads, GPU memory, input formats, and unexpected outputs: FAQ.

License

Model weights, code, and documentation are released under the Apache License 2.0.