nanovdr/ColNanoVDR-Q-Ettin150M-EVIE8B-4096-ML

ColNanoVDR: Document-Free Query-Side Distillation for Multi-Vector Visual Document Retrieval

Project page  |  Paper  |  Models  |  Code  |  Dataset

---

What is ColNanoVDR

ColNanoVDR replaces the multi-billion-parameter query encoder of a multi-vector visual document retriever with a small text-only encoder, and leaves the document side untouched. The student is trained by document-free distillation: it reproduces the teacher's query token embeddings under an optimal-transport objective (OTW), so no page image is ever encoded and no relevance label is used. At serving time the teacher's existing page index is scored by MaxSim unchanged, and with no vision tower on the query path a query can be encoded on CPU.

This tower targets tencent/EVIE-8B, EVIE's 8.4B flagship retriever with a single 4096-d projection. It keeps 94.0% of the teacher's NDCG@5 and 94.0% of its NDCG@10 on ViDoRe v3. For EVIE-4.5B, whose Prefix-MRL widths and HAC-compressed indexes give a much smaller index, use ColNanoVDR-Q-Ettin150M-EVIE45B-2048-ML.

This model

Every ColNanoVDR release is named:

ColNanoVDR-Q-{backbone}-{teacher}-{dim}-ML

A tower is only valid with the teacher it was distilled from. This one requires pages indexed by tencent/EVIE-8B. Towers for other teachers are listed on the nanovdr page.

Training

Dataset. nanovdr/NanoVDR-Train: 1.49M queries, 711K English plus 778K machine translations into five Latin-script European languages. Only the query text is used. EVIE-8B query embeddings are cached once, so the teacher never runs during training.

Loss: OTW. A query is a set of token vectors, and the student and the teacher tokenize differently, so there is no token-to-token correspondence. OTW treats each side as a weighted set on the unit sphere and minimises the entropic optimal-transport cost between them under c(s,t) = 1 - <s,t>, with a learned weight per student token:

L = <P, C>,   C_ij = 1 - <s_i, t_j>,   P = Sinkhorn(C; a = learned student weights, b = uniform)

Settings: eps = 0.05, 50 Sinkhorn iterations, AdamW one-cycle, peak LR 3e-4, 3% warmup, effective batch 1024 (128 per GPU x 2 GPUs x 4 accumulation), 10 epochs.

Architecture. Ettin encoder -> bias-free linear projection to 4096-d -> per-token L2 normalisation -> learned weight head (a linear layer over the pre-projection hidden states, softmax over the query's tokens). Each token is scaled by its weight, so the emitted vectors are deliberately not unit-length and their norms sum to 1 per query. Special tokens are excluded from scoring. 150M parameters.

Usage

Requires sentence-transformers>=6.0 and transformers>=5.0 (tested with 6.1 and 5.13.1). Retrieval runs in two stages: the teacher indexes your pages once, offline; ColNanoVDR answers every query online.

Step 1 (offline, once): index your pages with EVIE-8B

EVIE-8B ships its own Sentence Transformers integration:

import torch
from sentence_transformers import MultiVectorEncoder

teacher = MultiVectorEncoder("tencent/EVIE-8B", device="cuda",
                             model_kwargs={"torch_dtype": torch.bfloat16})
page_embeddings = teacher.encode_document(page_images, convert_to_numpy=False)   # (n_tokens, 4096) each

This gives the same page embeddings as EVIE's own colpali_engine code with 1024 visual tokens per page, the setting the numbers below were measured at.

Step 2 (online, per query): encode with ColNanoVDR

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("nanovdr/ColNanoVDR-Q-Ettin150M-EVIE8B-4096-ML", trust_remote_code=True)

q = model.encode_query(["What was the revenue growth in Q3 2024?"])
scores = model.similarity(q, page_embeddings)   # meanMaxSim

No teacher, no image processor, no GPU required on this path. Pass the raw query, with no instruction prefix, and do not re-normalise the output: the learned token weights are folded into the vector norms.

Performance

ViDoRe v3, the 8 public datasets, all query languages. Pages are encoded by EVIE-8B at 1024 visual tokens; the teacher columns score them with EVIE's own queries, the student columns with ColNanoVDR's, so all error is attributed to the query tower. Measured with our harness; EVIE-8B scores 66.44 NDCG@10 here against 66.75 in EVIE's own report.

Efficiency

License

Apache-2.0, matching the teacher, tencent/EVIE-8B (Apache-2.0). The query tower was distilled from EVIE-8B's query embeddings; please also credit EVIE (citation below). The backbone, Ettin, is MIT.

Contact

Zhuchenyang Liu, Aalto University: zhuchenyang.liu@aalto.fi

Citation

If you use ColNanoVDR, please cite:

@misc{liu2026colnanovdrdocumentfreequerydistillation,
      title={ColNanoVDR: Document-Free Query Distillation for Multi-Vector Visual Document Retrieval via Optimal Transport},
      author={Zhuchenyang Liu and Ziyi Wang and Yao Zhang and Yu Xiao},
      year={2026},
      eprint={2609.34899},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2609.34899},
}

The teacher:

@misc{tencent2026evie,
  title        = {EVIE: High-Performance Multilingual Visual Document Retrieval with Matryoshka Embeddings and Token Compression},
  author       = {Wang, Zifei and Wen, Wei},
  year         = {2026},
  howpublished = {\url{https://github.com/Tencent/EVIE}},
  note         = {Corresponding author: Wei Wen <jawnrwen@tencent.com>}
}