nfcorpus-finetuned-minilm

sentence-transformers/all-MiniLM-L6-v2, fine-tuned in-domain on NFCorpus (BEIR benchmark, medical/nutrition retrieval) with contrastive learning. Part of a two-part study — fine-tuning gains vs. baselines, and a VecStream ANN benchmark on real embeddings — documented in full at torinriley/hf-research.

Results (NFCorpus test split, 323 queries, held out of fine-tuning)

A modest, real gain over the zero-shot base model (NDCG@10 +4.1% relative, Recall@100 +11.2% relative) — small because the fine-tune itself is small (see Training below). The zero-shot baseline's NDCG@10 (0.319) matches published all-MiniLM-L6-v2 NFCorpus numbers in the MTEB/BEIR ecosystem (~0.317) within normal run-to-run variance, which is a useful sanity check on the eval pipeline.

Training

• Base model: sentence-transformers/all-MiniLM-L6-v2 (22.7M params).
• Data: NFCorpus train qrels, up to 3 positive (query, relevant-document) pairs sampled per query (seed 1234) to avoid overweighting heavily-judged queries — 7,253 training pairs.
• Loss: MultipleNegativesRankingLoss (in-batch-negative contrastive / InfoNCE-style), via sentence-transformers.
• 3 epochs, batch size 32, default sentence-transformers optimizer settings. 404s wall-clock on CPU. Final train loss: 2.576.
• No hard-negative mining, no hyperparameter sweep, one seed — this is a first iteration, not a tuned result. See the linked report for what a longer run / harder negatives would likely add.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("torinriley/nfcorpus-finetuned-minilm")
query_emb = model.encode("What foods help lower cholesterol?", normalize_embeddings=True)
doc_emb = model.encode("Oat bran and soluble fiber intake...", normalize_embeddings=True)
similarity = query_emb @ doc_emb

Limitations

• Fine-tuned and evaluated on one domain (NFCorpus/medical) with one base model and one seed — a point estimate, not a generalizable claim about fine-tuning gains on other domains.
• In-batch negatives only; no hard-negative mining.
• Not evaluated zero-shot on other BEIR datasets (no claim of cross-domain generalization).

Full methodology, the VecStream ANN benchmark on this model's embeddings, and discussion of limitations: REPORT.md.