A late-interaction visual document retriever built on Qwen3.5-4B. Each query and page becomes a set of 2048-d token vectors, and relevance is their MaxSim score.
4.5B parameters | 2048-d token vectors, Matryoshka prefixes from 64 to 2048 | BF16 | bidirectional attention
nDCG ×100 at 2048-d and 1,792 visual tokens per page, scored with mteb 2.26.3.
| Industrial | Finance EN | Finance FR | Pharma | Energy | Physics | Computer Science | HR | Mean |
|---|---|---|---|---|---|---|---|---|
| 59.27 | 72.49 | 56.09 | 70.85 | 72.08 | 53.29 | 82.86 | 70.21 | 67.14 |
| ArxivQA | DocVQA | InfoVQA | ShiftProject | SynAI | SynEnergy | SynGov | SynHealth | TabFQuAD | TAT-DQA | Mean |
|---|---|---|---|---|---|---|---|---|---|---|
| 93.18 | 68.08 | 94.65 | 96.05 | 100.00 | 98.63 | 99.26 | 100.00 | 97.40 | 83.68 | 93.09 |
| BioMedical Lectures | ESG Reports HL | ESG Reports | Economics Reports | Mean |
|---|---|---|---|---|
| 69.33 | 77.33 | 68.91 | 63.54 | 69.78 |
The per-task mteb result files are in results/.
[!NOTE] Use
transformers>=5.2.0. Older versions ignore the config'sis_causal: false, run the model with causal attention and give different embeddings, so Sentence Transformers won't load the model on them.
pip install "sentence-transformers[image]>=6.0.0"
from sentence_transformers import MultiVectorEncoder
model = MultiVectorEncoder("athrael-soju/ColQwen3.5-4.5B-v4")
queries = [
"What is the variable represented on the y-axis of the graph?",
"Total outlay is maximum in which year?",
]
images = [
"https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc1.jpg",
"https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc2.jpg",
]
query_embeddings = model.encode_query(queries)
image_embeddings = model.encode_document(images)
scores = model.similarity(query_embeddings, image_embeddings)
The projection was trained with Matryoshka prefixes at 64, 128, 256, 512, 1024 and 2048 dims. Keeping the first W dims of every token vector and renormalising each to unit length gives a W-d embedding at 1/32 to 1/2 of the index size. The results above are at 2048-d.
v4 comes from a multi-stage distillation recipe. A larger late-interaction teacher scores each training query against its positive page and mined hard negatives, and a reranker removes negatives that are actually relevant. Student seeds start from the previous release, train LoRA adapters for one epoch against the teacher's scores, and merge back into their start.
The full recipe and datasets will be shared soon.
Apache 2.0
model.safetensors 9.1 GB1_Dense/model.safetensors 10 MBQwen/Qwen3.5-4B