SauerkrautLM-ColQwen3-4b-v0.1
🥇 Best 128-dim Model in Large (3-5B) Category | Excellent Performance with Half the Memory
SauerkrautLM-ColQwen3-4b-v0.1 achieves 90.80 NDCG@5 on ViDoRe v1, making it the #2 overall among 128-dim models and the best in the Large (3-5B) category for ViDoRe v1.
🎯 Why Visual Document Retrieval?
Traditional OCR-based retrieval loses layout, tables, and visual context. Our visual approach:
- ✅ No OCR errors - Direct visual understanding
- ✅ Layout-aware - Understands tables, forms, charts
- ✅ End-to-end - Single model, no pipeline complexity
🏆 Key Achievements
Large Category Comparison (3-5B, 128-dim)
Best ViDoRe v1 in the Large category!
Detailed Benchmark Results
📊 ViDoRe v1 (NDCG@5) - Click to expand
📊 MTEB v1+v2 (NDCG@5) - Click to expand
ViDoRe v1 Tasks:
ViDoRe v2 Tasks (Multilingual):
📊 ViDoRe v3 (NDCG@10) - Click to expand
Overall Summary (128-dim Models)
📋 Summary Tables
128-dim Models Comparison
Comparison vs High-dim Models
✨ Key Features
- 🏆 #2 Overall (128-dim): Second highest ViDoRe v1 score among all 128-dim models
- 🥇 #1 in Large Category: Best 3-5B model on ViDoRe v1
- 💾 Memory Efficient: Only ~8GB VRAM (half of 8B model)
- ⚡ Compact Embeddings: 128-dimensional
- 🌍 Multilingual: 6 languages (EN, DE, FR, ES, IT, PT)
Model Details
Training
Hardware & Configuration
Datasets
Installation & Usage
Sentence Transformers
This model can be used with Sentence Transformers as a multi-vector (ColBERT-style late interaction) retriever via the MultiVectorEncoder:
pip install "sentence-transformers[image]>=6.0.0"
from sentence_transformers import MultiVectorEncoder
model = MultiVectorEncoder(
"VAGOsolutions/SauerkrautLM-ColQwen3-4b-v0.1",
model_kwargs={"dtype": "bfloat16"},
)
queries = [
"What is the variable represented on the y-axis of the graph?",
"Total outlay is maximum in which year?",
]
images = [
"https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc1.jpg",
"https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc2.jpg",
]
query_embeddings = model.encode_query(queries)
image_embeddings = model.encode_document(images)
print(query_embeddings[0].shape, image_embeddings[0].shape)
# torch.Size([25, 128]) torch.Size([1251, 128])
# Diagonal should have higher scores
scores = model.similarity(query_embeddings, image_embeddings)
print(scores)
# tensor([[16.3877, 8.1367],
# [ 5.8350, 15.3848]], device='cuda:0')
SauerkrautLM ColPali
⚠️ Important: Install our package first before loading the model:
pip install git+https://github.com/VAGOsolutions/sauerkrautlm-colpali
import torch
from PIL import Image
from sauerkrautlm_colpali.models import ColQwen3, ColQwen3Processor
model_name = "VAGOsolutions/SauerkrautLM-ColQwen3-4b-v0.1"
model = ColQwen3.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
attn_implementation="flash_attention_2",
device_map="cuda:0",
).eval()
processor = ColQwen3Processor.from_pretrained(model_name)
images = [Image.open("document.png")]
queries = ["What is the main topic?"]
batch_images = processor.process_images(images).to(model.device)
batch_queries = processor.process_queries(queries).to(model.device)
with torch.no_grad():
image_embeddings = model(**batch_images)
query_embeddings = model(**batch_queries)
scores = processor.score(query_embeddings, image_embeddings)
📊 Additional Benchmark Visualizations
MTEB v1+v2 Benchmark (128-dim Models)
ViDoRe v3 Benchmark (128-dim Models)
Our Models vs High-dim Models
Citation
@misc{sauerkrautlm-colpali-2025,
title={SauerkrautLM-ColPali: Multi-Vector Vision Retrieval Models},
author={David Golchinfar},
organization={VAGO Solutions},
year={2025},
url={https://github.com/VAGOsolutions/sauerkrautlm-colpali}
}
Contact
