VisDoc AI: Multimodal Visual Document Retrieval with Late Interaction & Quantization

VisDoc AI is a visual document retrieval engine built on Vision-Language Models (ColQwen2.5 / ColPali) and multi-vector late-interaction MaxSim scoring.

It indexes and searches complex visual documents—engineering datasheets, electrical schematics, PCB diagrams, financial balance sheets, commercial invoices, and stamped inspection certificates—without OCR-based text chunking.

• GitHub Repository: https://github.com/adityapande403/VisDoc-AI
• Base Architecture: vidore/colqwen2.5-v0.2 (Qwen/Qwen2.5-VL-3B-Instruct backbone)
• Output Representation: Multi-vector token embeddings with dimension $D = 128$ and unit L2-normalization.
• Scoring Function: MaxSim Late Interaction: $$S(Q, D) = \sum_{i=1}^{N_q} M_{q, i} \max_{j \in D} (q_i \cdot d_j)$$

---

Benchmark Results (Empirically Measured)

1. Retrieval Quality & Compression Retention (Held-Out Test Set)

2. 10,000-Page Scale Benchmark (5,000,000 Multi-Vectors)

Two-stage candidate filtering reduces 1,000-page latency from 253.26 ms to 10.69 ms (23.7x acceleration).

---

Quickstart

from visdoc import VisualDocumentRetriever

# Initialize with INT8 compressed index (100% NDCG@5 retention, 2x memory reduction)
retriever = VisualDocumentRetriever(index_format="int8")

# Ingest and index PDF document pages
retriever.index_documents(paths=["./datasheet.pdf"])

# Run late-interaction visual search
results = retriever.search(
    query="Which pin provides the 5.0 V analog sensor supply on the TX-4000?",
    top_k=5,
)

for result in results:
    print(f"Document: {result.document_id} | Page: {result.page_number} | Score: {result.score:.4f}")

---

Citation & Reference

@software{visdoc_ai_2026,
  author = {Aditya Pande},
  title = {VisDoc AI: Multimodal Visual Document Retrieval with ColPali, ColQwen and Late Interaction},
  url = {https://github.com/adityapande403/VisDoc-AI},
  year = {2026}
}