credit-risk-retriever-bge-small-india
A 33M-parameter sentence-transformer fine-tuned for dense retrieval over Indian consumer-credit regulation: CICRA 2005, RBI Master Directions (Credit Information Companies 2025, Credit Information Reporting 2025, IRACP 2025, Cards Issuance & Conduct 2025, NBFC Scale Based Regulation, Internal Ombudsman, Digital Lending 2025) and allied statutes (SARFAESI, DPDP 2023, Consumer Protection Act 2019).
Base model: BAAI/bge-small-en-v1.5.
Intended use
Given a compliance/regulatory question in English, retrieve the most relevant passages from Indian consumer-credit regulatory text. Best paired with a hybrid pipeline (BM25 + dense + reranking) but effective standalone.
Out of scope: non-English queries, jurisdictions other than India, general consumer chat, and questions outside credit regulation.
Training data
520 triplets built from an 11-document official corpus:
- Anchor: a compliance-analyst question synthesized per passage by an LLM (2 questions per chunk; short, varied phrasing).
- Positive: the source passage itself.
- Hard negative: the top-ranked neighbour retrieved by the deployed hybrid index that is not the positive - legally adjacent text, which forces the encoder to learn fine-grained distinctions between neighbouring sections.
The generation and training pipeline deliberately excludes all hand-curated evaluation questions, so held-out numbers below are uncontaminated.
Training procedure
| Setting | Value |
|---|---|
| Loss | MultipleNegativesRankingLoss (InfoNCE), triplet form |
| Learning rate | 2e-5, cosine decay, 10% warmup |
| Batch size | 64 |
| Epochs | 4 |
| Max sequence length | 320 tokens |
| Seed | 42 |
Training loss: 3.39 -> 1.55 over 32 optimizer steps.
Evaluation
Held-out set of 78 gold-labelled questions (each anchored to a specific corpus document). A hit at rank k means a chunk from the gold document appears in the top-k. Stock baseline measured in the same run, same seed, same encoding code path.
| Model | Recall@1 | Recall@5 | Recall@10 | MRR@5 |
|---|---|---|---|---|
| bge-small-en-v1.5 (stock) | 0.680 | 0.910 | 0.962 | 0.764 |
| this model | 0.731 | 0.910 | 0.974 | 0.807 |
Recall@1 improves by +7.5% relative over stock and reaches parity with BM25 lexical retrieval on the same corpus (BM25: Recall@1 0.731 / Recall@10 0.923 / MRR@5 0.791), while dense retrieval extends its lead at k>=5 (Recall@10 +5.1 pts over BM25).
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("vivekkopthsd/credit-risk-retriever-bge-small-india")
model.max_seq_length = 320 # keep consistent with training
query = "Within how many days must a CIC complaint be resolved?"
docs = [
"Complaints shall be resolved within thirty calendar days ...",
"A credit card closure request must be honoured within seven working days ...",
]
q_emb = model.encode(["query: " + query], normalize_embeddings=True)
d_emb = model.encode(["passage: " + d for d in docs], normalize_embeddings=True)
scores = q_emb @ d_emb.T # higher = more relevant
Prefix conventions from training: query: for questions, passage: for documents.
Set max_seq_length = 320 at load time.
Limitations
- English only; Hindi-language regulations were not part of training.
- Domain-specific: tuned on Indian credit regulation; expect degraded quality on unrelated domains (general web text performed worse than stock in analogous runs).
- Trained on ~520 pairs - gains are documented at these depths rather than claimed as universally better than MS MARCO pretraining.