🔎 pplx-embed-v2-late-0.6b-ko-vdr

pplx-embed-v2-late-0.6b-ko-vdr is a 0.6B late-interaction (multi-vector) retriever for Korean visual document retrieval: it retrieves document page screenshots for text queries. It is fine-tuned from perplexity-ai/pplx-embed-v2-late-0.6b and improves nDCG@10 over the base model on Korean benchmarks (KoViDoRe, SDS-KoPub-VDR, MIRACL-VISION Korean subset) as well as on ViDoRe v3.

Key Characteristics

• Late interaction (ColBERT-style): queries and pages are encoded into sequences of 128-dimensional token vectors and scored with MaxSim.
• Visual document retrieval: documents are page images (screenshots of PDFs, reports, slides), so no OCR or layout parsing is needed.
• Compact: 0.6B parameters.
• Input lengths: up to 1,024 query tokens and 4,096 document tokens.

Usage

pip install -U sentence-transformers

Use encode_query / encode_document: they add the [Q] / [D] prompts the model was trained with.

from sentence_transformers import MultiVectorEncoder

# Download from the 🤗 Hub
model = MultiVectorEncoder("MarkrAI/pplx-embed-v2-late-0.6b-ko-vdr")
# Run inference: each input becomes a sequence of per-token vectors (variable length).
queries = [
    '30인 이상 상용근로자를 보유한 기업의 1인당 평균 월별 법정외 복지비용이 10~29인 규모 기업보다 높은지 판단해 주세요.',
]
documents = [
    'path/to/page_1.png',  # page screenshot (file path, URL, or PIL.Image)
    'path/to/page_2.png',  # page screenshot (file path, URL, or PIL.Image)
    'path/to/page_3.png',  # page screenshot (file path, URL, or PIL.Image)
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings[0].shape, document_embeddings[0].shape)
# e.g. (40, 128) (1718, 128)

# Get the MaxSim similarity scores
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)

Note: The base model uses causal_conv1d CUDA kernels, so inference requires a GPU.

Evaluation

Evaluated with MTEB. We report nDCG@10.

Per-task nDCG@10

KoViDoRe

ViDoRe v3 (public)

Citation

If you use this model, please cite:

@misc{markrai2026pplxembedv2late06bkovdr,
  author = {{Marker-Inc-Korea} and Yongbin Choi},
  title = {{pplx-embed-v2-late-0.6b-ko-vdr: A Late-Interaction Retriever for Korean Visual Document Retrieval}},
  year = {2026},
  url = {https://huggingface.co/MarkrAI/pplx-embed-v2-late-0.6b-ko-vdr}
}

Base Model

@online{perplexity2026MultimodalEmbeddingsBeyondASingleVector,
  author = {{Perplexity} and Markus Krimmel and Louis Milliken and Antoine Chaffin and Sarah Eslami and Ekaterina Glazkova and Maximilian Schall and Bo Wang and Denis Bykov and Jarrod Bryan and Claire Beamer and Jeremy Yang},
  title = {{Multimodal embeddings beyond a single vector}},
  date = {2026-10-07},
  url = {https://www.perplexity.ai/hub/blog/multimodal-embeddings-beyond-a-single-vector}
}

KoViDoRe

@inproceedings{choi2026kovidore,
  title={KoViDoRe: A Benchmark for Korean Visual Document Retrieval},
  author={Choi, Yongbin and Song, Yongwoo and Sung, Mujeen},
  booktitle={Proceedings of the 2nd Workshop on Multimodal Augmented Generation via Multimodal Retrieval (MAGMaR 2026)},
  pages={54--80},
  year={2026}
}