VultronRetrieverPrime v2 is the second release of the flagship of the VultronRetriever family, a late-interaction (ColBERT-style) retriever that scores document pages directly from their rendered image: layout, tables, charts and text, across six languages. v2 moves from 320 to 4096-dimensional multi-vectors and runs its full-attention layers bidirectionally.
The family has three tiers: the 8B Prime (this model) for maximum accuracy, the 4.5B Core for the accuracy/footprint mid-point, and the 0.8B Flash for latency- and footprint-sensitive serving. Trained and evaluated on Vultr Cloud.
Scores are nDCG ×100 from mteb 2.26.3, using 4096-dim embeddings and up to 1,792 visual tokens per page.
| Industrial | Finance EN | Finance FR | Pharma | Energy | Physics | Computer Science | HR | Mean |
|---|---|---|---|---|---|---|---|---|
| 59.65 | 72.62 | 57.04 | 70.96 | 73.03 | 53.54 | 82.84 | 70.00 | 67.46 |
| ArxivQA | DocVQA | InfoVQA | ShiftProject | SynAI | SynEnergy | SynGov | SynHealth | TabFQuAD | TAT-DQA | Mean |
|---|---|---|---|---|---|---|---|---|---|---|
| 93.53 | 70.37 | 95.36 | 95.55 | 99.63 | 97.63 | 99.26 | 100.00 | 97.74 | 84.46 | 93.35 |
| BioMedical Lectures | ESG Reports HL | ESG Reports | Economics Reports | Mean |
|---|---|---|---|---|
| 70.98 | 79.24 | 73.08 | 68.35 | 72.91 |
Per-task mteb result files are in results/. The 2 private V3 tasks are scored by the ViDoRe maintainers.
The token vectors can be shortened: keep the first N dimensions of each vector and re-normalize it. The index shrinks in proportion. Scores are nDCG ×100 on the full benchmarks at each size.
| Dim | Index size | V1 | V2 | V3 (8 public) |
|---|---|---|---|---|
| 4096 | 1× | 93.35 | 72.91 | 67.46 |
| 2048 | 1/2 | 93.28 | 72.85 | 67.37 |
| 1024 | 1/4 | 93.29 | 73.06 | 67.17 |
| 512 | 1/8 | 93.14 | 73.13 | 66.79 |
| 256 | 1/16 | 92.73 | 71.80 | 66.40 |
ViDoRe V3 per task (nDCG@10):
| Dim | Industrial | Finance EN | Finance FR | Pharma | Energy | Physics | Computer Science | HR | Mean |
|---|---|---|---|---|---|---|---|---|---|
| 4096 | 59.65 | 72.62 | 57.04 | 70.96 | 73.03 | 53.54 | 82.84 | 70.00 | 67.46 |
| 2048 | 59.68 | 72.47 | 57.07 | 70.86 | 72.98 | 53.41 | 82.64 | 69.90 | 67.37 |
| 1024 | 59.15 | 72.43 | 56.99 | 70.57 | 72.64 | 53.33 | 82.72 | 69.54 | 67.17 |
| 512 | 58.63 | 72.06 | 57.02 | 70.08 | 72.28 | 52.92 | 82.52 | 68.77 | 66.79 |
| 256 | 58.23 | 71.41 | 56.87 | 69.22 | 72.22 | 52.48 | 82.21 | 68.56 | 66.40 |
Top 10 by mean nDCG@10 over the 8 public ViDoRe V3 tasks; V1, V2 and the 10-task V3 mean are shown alongside.
| Model | Params | Dim | V1 | V2 | V3 (8 public) | V3 (all 10) |
|---|---|---|---|---|---|---|
| VultronRetrieverPrime-Qwen3.5-8B-v2 (this model) | 8.4B | 4096 | 93.35 | 72.91 | 67.46 | pending |
| athrael-soju/ColQwen3.5-4.5B-v4 | 4.5B | 2048 | 93.09 | 69.78 | 67.14 | pending |
| tencent/EVIE-8B | 8.4B | 4096 | 92.90 | 74.12 | 66.45 | 66.24 |
| Microsoft/microsoft-embedding-1-large | — | 1024 | — | — | 66.24 | — |
| tencent/EVIE-4.5B | 4.5B | 2048 | 92.86 | 72.92 | 65.88 | 65.70 |
| webAI-Official/webAI-ColVec1.1-8b | 8.4B | 640 | 91.53 | 67.08 | 65.32 | 64.95 |
| perplexity-ai/pplx-embed-v2-late-9b | 8.4B | 128 | 90.67 | 68.96 | 64.95 | 64.77 |
| VultronRetrieverPrime-Qwen3.5-8B (v1) | 8.4B | 320 | 92.08 | 68.18 | 64.72 | 64.26 |
| webAI-Official/webAI-ColVec1-9b | 9.4B | 2560 | 91.30 | 65.82 | 64.45 | 63.00 |
| webAI-Official/webAI-ColVec1.1-4b | 4.5B | 640 | 90.91 | 66.60 | 64.24 | 63.90 |
V1/V2: mean nDCG@5. V3 (8 public): mean nDCG@10 over the 8 public ViDoRe V3 tasks. V3 (all 10) adds the 2 private tasks (Nuclear, Telecom), scored by the ViDoRe maintainers on held-out corpora. Other models' figures: MTEB results repository, 2026-11-18; "—" = not reported on that benchmark.
* = best on that task among the models listed. Mean is over the 8 public tasks; Nuc and Tel are the private tasks.
| Model | CS | Energy | FinEn | FinFr | Hr | Ind | Nuc | Pharma | Phys | Tel | Mean |
|---|---|---|---|---|---|---|---|---|---|---|---|
| VultronRetrieverPrime-Qwen3.5-8B-v2 (this model) | 82.8 | 73.0* | 72.6* | 57.0* | 70.0 | 59.6* | pending | 71.0 | 53.5* | pending | 67.46 |
| athrael-soju/ColQwen3.5-4.5B-v4 | 82.9* | 72.1 | 72.5 | 56.1 | 70.2* | 59.3 | pending | 70.8 | 53.3 | pending | 67.14 |
| tencent/EVIE-8B | 82.1 | 72.2 | 71.2 | 55.8 | 68.9 | 59.1 | 56.6* | 70.3 | 52.0 | 74.3* | 66.45 |
| Microsoft/microsoft-embedding-1-large | 82.6 | 72.5 | 69.4 | 54.5 | 68.5 | 59.3 | — | 71.2* | 51.9 | — | 66.24 |
| tencent/EVIE-4.5B | 82.0 | 72.0 | 69.9 | 54.7 | 67.6 | 58.8 | 56.4 | 70.0 | 52.1 | 73.5 | 65.88 |
| webAI-Official/webAI-ColVec1.1-8b | 80.1 | 70.1 | 71.9 | 54.9 | 68.5 | 57.6 | 53.7 | 67.9 | 51.5 | 73.3 | 65.32 |
| perplexity-ai/pplx-embed-v2-late-9b | 79.5 | 71.7 | 71.6 | 53.9 | 67.7 | 56.2 | 54.5 | 68.9 | 50.1 | 73.6 | 64.95 |
| VultronRetrieverPrime-Qwen3.5-8B (v1) | 79.8 | 70.3 | 69.0 | 54.5 | 66.8 | 57.4 | 53.6 | 68.2 | 51.7 | 71.3 | 64.72 |
| webAI-Official/webAI-ColVec1-9b | 80.9 | 69.8 | 68.3 | 53.7 | 70.0 | 57.2 | 47.7 | 67.3 | 48.4 | 66.7 | 64.45 |
| webAI-Official/webAI-ColVec1.1-4b | 80.3 | 69.5 | 69.2 | 53.1 | 66.9 | 56.4 | 53.3 | 67.2 | 51.2 | 71.8 | 64.24 |
Out of scope: text-only semantic search, where a single-vector dense embedder is cheaper; generative QA (this is a retriever; pair it with a reader/LLM).
Per-token MaxSim scoring captures fine-grained matches against tables, figures, and layout that a single-vector embedder averages away.
Qwen/Qwen3.5-9B (hybrid GatedDeltaNet + full-attention backbone).ColQwen3_5): 4096-dim multi-vector embeddings, MaxSim scoring,
image + text inputs.An enhanced, multilingual mixture of public and synthetic visual-document retrieval sources, spanning en, es, de, fr, it and pt. The training recipe and the assembled training dataset are not distributed in this repository.
[!NOTE] Use
transformers>=5.2.0. Older versions ignore the config'sis_causal: false, run the model with causal attention and give different embeddings, so Sentence Transformers won't load the model on them.
This model can be used with Sentence Transformers as a multi-vector (ColBERT-style late interaction) retriever via the MultiVectorEncoder:
pip install "sentence-transformers[image]>=6.0.0"
from sentence_transformers import MultiVectorEncoder
model = MultiVectorEncoder("vultr/VultronRetrieverPrime-Qwen3.5-8B-v2")
queries = [
"What is the variable represented on the y-axis of the graph?",
"Total outlay is maximum in which year?",
]
images = [
"https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc1.jpg",
"https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc2.jpg",
]
query_embeddings = model.encode_query(queries)
image_embeddings = model.encode_document(images)
# Diagonal should have higher scores
scores = model.similarity(query_embeddings, image_embeddings)
print(scores)
config.json carries dim=4096, so the projection is sized correctly at load. It also carries
text_config.is_causal=false and retrieval_attention_contract="bidirectional".
vLLM serves this model natively through its pooling runner (the ColQwen3_5 architecture), returning
the per-token multi-vectors for late-interaction scoring. It requires a vLLM build that includes the
projection-bias fix and explicit retrieval-attention metadata support. The handler reads
retrieval_attention_contract="bidirectional" from this checkpoint and runs all eight full-attention
layers bidirectionally; it must reject missing, conflicting, or unsupported contract
metadata. The server uses the stock chat/image processor, so the ColQwen3.5
prompt contract is applied client-side: wrap each page image in the instruction template, append
the query-augmentation tokens to each query, and set the visual-token budget through
mm-processor-kwargs. Prefix caching and chunked prefill remain off for the GatedDeltaNet hybrid.
pip install vllm
import torch
from PIL import Image
from vllm import LLM
MODEL = "vultr/VultronRetrieverPrime-Qwen3.5-8B-v2"
MAX_PIXELS = 1792 * 32 * 32 # max_num_visual_tokens * (patch_size 16 * merge_size 2)^2
llm = LLM(
model=MODEL,
runner="pooling",
dtype="bfloat16",
enable_prefix_caching=False,
enable_chunked_prefill=False,
mm_processor_kwargs={"min_pixels": 65536, "max_pixels": MAX_PIXELS},
)
# ColQwen3.5 processor contract, applied client-side:
IMAGE_PROMPT = ("<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|>"
"Describe the image.<|im_end|><|endoftext|>")
def query_prompt(q): return q + "<|endoftext|>" * 10 # query augmentation
images = [Image.open("page_0.png"), Image.open("page_1.png")]
queries = ["What was Q3 revenue?", "Summarize the safety findings."]
doc_out = llm.encode([{"prompt": IMAGE_PROMPT, "multi_modal_data": {"image": im}}
for im in images], pooling_task="token_embed")
qry_out = llm.encode([query_prompt(q) for q in queries], pooling_task="token_embed")
def mv(o): # one [num_tokens, 4096] multi-vector per item, L2-normalized per token
t = torch.as_tensor(o.outputs.data, dtype=torch.float32)
return torch.nn.functional.normalize(t, p=2, dim=-1)
docs, qrys = [mv(o) for o in doc_out], [mv(o) for o in qry_out]
# late-interaction MaxSim: per query token take the best doc token, then sum
scores = [[(q @ d.T).max(dim=-1).values.sum().item() for d in docs] for q in qrys]
print(scores) # scores[i][j] = relevance of query i to page j
To serve over HTTP instead:
vllm serve vultr/VultronRetrieverPrime-Qwen3.5-8B-v2 \
--runner pooling \
--no-enable-prefix-caching --no-enable-chunked-prefill \
--mm-processor-kwargs '{"min_pixels": 65536, "max_pixels": 1835008}'
Apply the same image template and query augmentation in your client requests. See the upstream example
examples/pooling/score/colqwen3_5_rerank_online.py for the full online rerank flow.
Apache 2.0, covering the contents of this repository: model weights, config, and evaluation results.
Built on Qwen/Qwen3.5-9B (Apache 2.0); the upstream license and attribution are retained. The
training recipe and the assembled training dataset are not distributed in this repository.
@misc{vultronretrieverprime2026v2,
title = {VultronRetrieverPrime-Qwen3.5-8B-v2: Late-Interaction Visual Document Retrieval},
author = {Georgiou, Athos (athrael-soju)},
year = {2026},
howpublished = {\url{https://huggingface.co/vultr/VultronRetrieverPrime-Qwen3.5-8B-v2}}
}
Trained and evaluated on Vultr Cloud.
model.safetensors 17 GB1_Dense/model.safetensors 34 MBQwen/Qwen3.5-9B