FlipAware_all_MiniLM_L6_v2_smoke

sentence-transformers/all-MiniLM-L6-v2 fine tuned to tell apart sentences that look alike but mean different things, while keeping its normal search quality. It is a drop in replacement for the base model: same architecture, same vector size, same prompts.

Dataset: FlipAware_Triples_smoke | Leaderboard: FlipAware Leaderboard | Logs: FlipAware Runs | Code: GitHub

Results

Trap accuracy is the share of triples where the true paraphrase scores closer to the anchor than the look alike trap. Tuned numbers are mean ± std over 1 seed(s). Test sentences never appear in training.

Verdict: GOOD. Trap accuracy +0.0 points, retrieval nDCG@10 -3.4 points, cross generator +0.0 points.

Confidence

Paired bootstrap over test triples, best seed against the base model.

Accuracy by trap type

Shorter and quantized vectors

Ablation

mined guard trains only on the triples this base model got wrong, plus a sample of easy ones.

How much data it takes

Blending tuned and base weights

Weights blended as (1 - alpha) * base + alpha * tuned, with no extra training.

Best blend inside the retrieval budget: alpha 0.

Where embedders fail

Compared with other open embedders

Full table: FlipAware Leaderboard

Training curves

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("AmareshHebbar/FlipAware_all_MiniLM_L6_v2_smoke")

a, b = model.encode(["The drug is safe for children.", "The drug is not safe for children."])
print(model.similarity(a, b))

queries = model.encode(["is the drug safe for kids"])
docs = model.encode(["The drug is not safe for children under 12."])
print(model.similarity(queries, docs))

Shorter vectors: pass truncate_dim=256 (or any trained size) when loading.

Vectors from this model are not compatible with vectors from the base model. Re embed your corpus after switching.

Training

• Data: FlipAware_Triples_smoke, anchor, paraphrase, and look alike trap across 8 trap types, plus Natural Questions retrieval pairs so search skill does not decay.
• Loss: CachedMultipleNegativesRankingLoss inside MatryoshkaLoss, with the trap as a hard negative.
• Guard: CosineSimilarityLoss pulling similarity on general sentence pairs back to the base model's own scores.
• Matryoshka dims: 384, 256, 128, 64
• Epochs 1, batch 32 (mini batch 16), learning rate 2e-05, max length 128, precision fp16, LoRA no
• Hardware: Tesla T4, 0.05 GPU hours for all runs of this base model
• Training curves: the Training metrics tab and assets/curve.png.
• Every log and metric: FlipAware Runs, live in TensorBoard and MLflow.
• Code: https://github.com/amareshhebbar/flipaware

Prior work

NevIR (Weller et al., 2024) showed that neural retrievers fail when two documents differ only by negation. This model trains a fix across eight trap types, checks it on human written and cross generator test sets, and reports general retrieval next to it, so any regression is visible.

Limitations

• English only.
• Training triples are written by Qwen/Qwen3-4B-Instruct-2507, which belongs to the same family as the Qwen3 embedders. The cross generator and human test sets exist to check for that bias.
• Eight trap types; other meaning flips are not covered.
• NanoBEIR is a small sample of BEIR, used as a quick check for general retrieval.