bge-small-rrf-v2: A 33M Parameter Model That Beats ColBERTv2 on 3/5 BEIR Datasets

Paper: vstash: Local-First Hybrid Retrieval with Adaptive Fusion for LLM Agents (arXiv:2604.15484)

Trained with zero human labels using a novel self-supervised signal: hybrid retrieval disagreement.

When vector search and keyword search disagree on what's relevant for a query, that disagreement reveals where the embedding model fails. We exploit this signal to fine-tune BGE-small, producing a model that better distinguishes "semantically close" from "actually relevant."

Key Result

A 33M parameter model, fine-tuned for $0 with zero human labels, surpasses ColBERTv2 (110M parameters) on 3 out of 5 standard BEIR benchmarks:

Up to +19.5% NDCG improvement over the base model, with zero additional inference cost.

Why This Matters

Most embedding improvements require either:

• A larger model (more compute, more latency)
• Human-labeled training data (expensive, slow)
• A teacher model for distillation (adds complexity)

This model needs none of that. The training signal comes from running the existing hybrid retrieval pipeline and observing where its two components (vector search and keyword search) disagree. The system improves itself.

The Training Signal: Hybrid Retrieval Disagreement

We discovered that 82% of queries produce disagreement between vector and keyword search in the top-5 results. These disagreements fall into two categories:

• Vector blind spots (51%): chunks the vector search ranks high but keyword search ignores. These are semantically similar but not actually relevant.
• Keyword blind spots (49%): chunks keyword search finds but vector search misses. These contain relevant terms but the embedding doesn't recognize their relevance.

Fine-tuning on these disagreement pairs teaches the model to fix both types of blind spots.

Training Details

Why MNRL, not TripletLoss?

We tested TripletLoss first. It destroyed the model (-84% NDCG after 3 epochs). TripletLoss pushes individual negatives away with brute force, distorting the embedding space. MNRL adjusts relationships across 64 documents simultaneously per batch, preserving the model's general knowledge while learning from disagreements.

Usage

With sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Stffens/bge-small-rrf-v2")
embeddings = model.encode(["your query", "your document"])
similarity = embeddings[0] @ embeddings[1]

With vstash (hybrid retrieval system)

pip install vstash
vstash reindex --model Stffens/bge-small-rrf-v2
vstash search "your query"

Train your own version on your data

pip install vstash sentence-transformers torch
vstash retrain  # generates disagreement pairs from YOUR corpus and fine-tunes
vstash reindex --model ~/.vstash/models/retrained

Reproduce From Scratch

git clone https://github.com/stffns/vstash
cd vstash
pip install -e . sentence-transformers torch

# Generate disagreement triples
python -m experiments.rrf_training_pairs --datasets scifact nfcorpus fiqa

# Train (GPU recommended)
python -m experiments.finetune_rrf --epochs 2 --lr 3e-6 --batch-size 64

# Evaluate
python -m experiments.finetune_rrf --evaluate-only

Limitations

• ArguAna regression: queries with 200+ words show -8.4% vs ColBERTv2. Long argumentative queries produce only 1.1% signal disagreement, leaving no training signal.
• FiQA neutral: financial queries show +0.1% vs base but -7.8% vs ColBERTv2. The disagreement signal exists (86.7%) but doesn't translate to NDCG gains on this dataset.
• English only: inherited from BGE-small-en-v1.5.
• Not tested beyond BEIR: performance on domain-specific corpora may vary.

Citation

@software{vstash2026,
  author = {Steffens, Jayson},
  title = {vstash: Local-First Hybrid Retrieval with Adaptive Fusion for LLM Agents},
  url = {https://github.com/stffns/vstash},
  year = {2026}
}

Related

• vstash paper (Section 8.10: Self-Supervised Embedding Refinement)
• vstash GitHub
• Base model: BAAI/bge-small-en-v1.5