WeVisDoc-4B (GGUF)
GGUF quantizations of tencent/WeVisDoc-4B — Tencent's 4B WeVisDoc, an end-to-end document parser fine-tuned from Qwen3-VL that turns a page image into structured Markdown, with LaTeX formulas and HTML tables — optimized for local inference with llama.cpp and Ollama, with its vision projector. All quants use an importance matrix.
Quick Start
With Ollama
ollama run richardyoung/wevisdoc-4b
# or straight from Hugging Face
ollama run hf.co/richardyoung/WeVisDoc-4B-GGUF:Q4_K_M
With llama.cpp
huggingface-cli download richardyoung/WeVisDoc-4B-GGUF \
--include "*Q4_K_M*" "mmproj*" --local-dir ./models
./llama-mtmd-cli -m ./models/WeVisDoc-4B-Q4_K_M.gguf \
--mmproj ./models/mmproj-WeVisDoc-4B-F16.gguf \
--image page.png -p "Parse this document page into Markdown."
Prompts: send a page image and ask for Markdown (e.g. Parse this document page into Markdown.). Tencent's own client and its exact prompt are in the WeVisDoc repository.
Why This Model?
- OmniDocBench v1.6 Overall 95.38, first among the end-to-end parsers Tencent compared
- Page image in, Markdown out: formulas as LaTeX, tables as HTML
- English and Chinese
- Apache-2.0
Use Cases
- Converting scanned pages and PDFs (rendered to images) into Markdown
- Formula-heavy papers (LaTeX output)
- Tables to HTML for RAG pipelines
Available Quantizations
| File | Size | Use Case |
|---|---|---|
WeVisDoc-4B-IQ3_M.gguf |
2.1 GB | Smallest |
WeVisDoc-4B-IQ4_XS.gguf |
2.5 GB | Great quality-size balance |
WeVisDoc-4B-Q4_K_M.gguf |
2.7 GB | Recommended — good balance |
WeVisDoc-4B-Q5_K_M.gguf |
3.2 GB | Higher quality |
WeVisDoc-4B-Q6_K.gguf |
3.6 GB | Very high quality |
WeVisDoc-4B-Q8_0.gguf |
4.7 GB | Near-lossless |
mmproj-WeVisDoc-4B-F16.gguf |
0.84 GB | Vision projector (required for images) |
WeVisDoc-4B-imatrix.gguf |
– | Importance matrix used for the quants (calibration_datav3) |
Quant Quality
Each quant compared with the F16 model using llama-perplexity --kl-divergence on the wikitext-2 test set (40 × 512-token chunks). KL divergence measures how far the quant's next-token probabilities drift from F16; lower is better. Top-token match is how often the quant picks the same next token as F16. As a rough guide, a KLD under 0.01 is hard to tell from F16, and from about 0.05 differences start to show in longer outputs. These numbers cover the text decoder only, on English prose. For OCR accuracy, see Verified below.
| Quant | Size | KL divergence | Top-token match | Perplexity (F16: 11.47) |
|---|---|---|---|---|
| IQ3_M | 2.1 GB | 0.3594 | 74.0% | 13.77 |
| IQ4_XS | 2.5 GB | 0.1279 | 83.8% | 12.06 |
| Q4_K_M | 2.7 GB | 0.1011 | 86.1% | 11.72 |
| Q5_K_M | 3.2 GB | 0.0406 | 91.0% | 11.61 |
| Q6_K | 3.6 GB | 0.0218 | 93.4% | 11.52 |
| Q8_0 | 4.7 GB | 0.0042 | 97.1% | 11.44 |
Verified
Tested through Ollama before upload (Q4_K_M and Q8_0) on a synthetic invoice with the prompt Parse this document page into Markdown.; Q4_K_M returned: '## INVOICE 4471 / Date: 2026-09-28 / Widgets x3 ..... $42.00 / TOTAL: $86.20'.
Also Available
- WeVisDoc-2B GGUF — smaller and faster
Other Models by richardyoung
- New Heretic models (reproducible, listed in the Heretic Grimoire): LFM2.5-1.2B | Granite-4.2-3B | Granite-4.2-8B | Olmo-3-7B-Think | Phi-4-mini | DeepSeek-R1-0528-Qwen3-8B | Agents-A1-4B | FrogNano-4B | Muse-Glimmer-30B | Hemmingway-1
- Abliterated/Uncensored models: Qwen2.5-7B | Qwen3-14B | DeepSeek-R1-32B | Qwen3-8B
- OCR & Vision: MiniCPM-V-4.6 GGUF | dots.mocr GGUF | Unlimited-OCR GGUF | olmOCR GGUF | LightOnOCR-3-4B GGUF | WeVisDoc-4B GGUF | Xiaomi-OCR-0 GGUF
- MLX quantizations (Apple Silicon): Kimi-K2 series | olmOCR MLX
- Healthcare/Medical: Synthea 575K patients dataset | CardioEmbed
- Research: LLM Instruction-Following Evaluation (arxiv:2510.18892)
Built & maintained by Richard Young · DeepNeuro