BiUltraVBERT

BiUltraVBERT is a 255M-parameter single-vector retriever for visual documents, fine-tuned from UltraVBERT. It encodes each query and each page as a single 768-d vector and ranks pages by cosine similarity. Truncating the vectors to 128, 256, 320 or 512 dimensions gives a smaller index.

Models

Data

Usage

Requires sentence-transformers>=6.1 and transformers>=5.21, which supports UltraVBERT natively (docs). On 5.20.x, pass trust_remote_code=True to load the code in this repository.

from PIL import Image
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("athrael-soju/BiUltraVBERT")

pages = [Image.open("page1.png"), Image.open("page2.png")]
queries = ["What was the 2023 revenue?", "Which table lists regional margins?"]

page_embeddings = model.encode_document(pages)
query_embeddings = model.encode_query(queries)

# One row per query, one column per page.
scores = model.similarity(query_embeddings, page_embeddings)

For a smaller index, load the model with truncate_dim set to 128, 256, 320 or 512.

Model

BiUltraVBERT mean-pools UltraVBERT's token states, projects the result to 768 dimensions with a linear layer and L2-normalizes it. Pages are resized to at most 3,072 px on the longest edge and split into up to 25 tiles of 512 px, plus a downscaled view of the whole page.

Training

The released weights are a per-block merge of four fine-tunes of UltraVBERT on VDR Train 500k that differ only in seed. Each fine-tune trains rank-32 LoRA adapters on the text encoder and the output layer for one epoch at batch size 256, with Matryoshka losses at 128, 256, 320, 512 and 768 dimensions; the vision encoder and connector stay frozen.

ViDoRe

BiUltraVBERT's scores are its official MTEB results (results#785): MTEB 2.26.10, 768-d vectors, bf16 and SDPA attention. Multilingual tasks report the mean over their language splits. The MTEB maintainers scored the private ViDoRe V3 tasks, Nuclear and Telecom (results#791). BiModernVBERT's V1 and V2 scores are our MTEB 2.26.3 runs of ModernVBERT/bimodernvbert at revision 09ed8566; its V3 scores are its MTEB results.

Smaller indexes

Scores come from cached page features; at 768 dimensions they are within 0.25 of the official means.

License

MIT. The UltraBERT-150M and SigLIP2-B/16-512 components are Apache-2.0, with their license text in LICENSE-APACHE.