This repository hosts the GGUF build of MiniCPM-OCR Jev for llama.cpp. For the original weights and the full model card, see the base repository.
MiniCPM-OCR Jev (2B) scores OCR candidates for text, formula, and table regions, returning a quality score from 0 to 1.
Files
| File | Size | Notes |
|---|---|---|
mmproj-model-f16.gguf |
0.671 GB | vision projector — required |
MiniCPM-OCR-Jev-F16.gguf |
3.776 GB | reference |
MiniCPM-OCR-Jev-Q8_0.gguf |
2.012 GB | near-lossless |
MiniCPM-OCR-Jev-Q6_K.gguf |
1.556 GB | |
MiniCPM-OCR-Jev-Q5_K_M.gguf |
1.411 GB | |
MiniCPM-OCR-Jev-Q5_K_S.gguf |
1.374 GB | |
MiniCPM-OCR-Jev-Q5_1.gguf |
1.460 GB | |
MiniCPM-OCR-Jev-Q5_0.gguf |
1.374 GB | |
MiniCPM-OCR-Jev-Q4_K_M.gguf |
1.274 GB | smallest recommended |
MiniCPM-OCR-Jev-Q4_K_S.gguf |
1.212 GB | |
MiniCPM-OCR-Jev-Q4_1.gguf |
1.288 GB | |
MiniCPM-OCR-Jev-Q4_0.gguf |
1.202 GB | |
MiniCPM-OCR-Jev-IQ4_XS.gguf |
1.202 GB | imatrix |
MiniCPM-OCR-Jev-IQ4_NL.gguf |
1.236 GB | imatrix |
MiniCPM-OCR-Jev-IQ3_M.gguf |
1.059 GB | imatrix |
MiniCPM-OCR-Jev-IQ3_XS.gguf |
1.027 GB | imatrix |
Usage
llama-server \
--model ./MiniCPM-OCR-Jev-Q8_0.gguf \
--mmproj ./mmproj-model-f16.gguf \
--host 0.0.0.0 --port 18498 \
--ctx-size 40960 \
--n-gpu-layers 999
Check verdict in the response — only scored means scoring actually ran.
Deployment details: llama.cpp guide for Jev.
Related checkpoints
| Format | Hugging Face | ModelScope |
|---|---|---|
| BF16 (original) | 🤗 | ModelScope |
| GGUF (llama.cpp) (this repo) | 🤗 | ModelScope |
| AWQ W4A16 | 🤗 | ModelScope |
| GPTQ W4A16 | 🤗 | ModelScope |
FAQ
Installation, model downloads, GPU memory, input formats, and unexpected outputs: FAQ.
License
Model weights, code, and documentation are released under the Apache License 2.0.