Dhara reranker: Indian criminal law
A cross-encoder that scores how well a passage from a Supreme Court of India criminal judgment, or a section of the criminal codes, answers a legal question. It is bge-reranker-base fine-tuned on question–passage pairs mined from the Dhara corpus, and it is the reranker in the Dhara research tool, live at dhara.aravarun.in.
It was built for the 1 July 2024 change of codes (IPC, CrPC and the Evidence Act replaced by BNS, BNSS and BSA). Questions are asked with new-code section numbers, but most judgments cite the old ones, so the model has to judge a passage on what it says, not on the number it quotes.
Use
from sentence_transformers import CrossEncoder
model = CrossEncoder("AravArun/dhara-reranker", max_length=512)
question = "Can a conviction under BNS 103 rest on circumstantial evidence alone?"
passages = [
"Where the case rests on circumstantial evidence, the chain must be so complete that it excludes every hypothesis but guilt...",
"The appellant was granted anticipatory bail subject to conditions...",
]
scores = model.predict([(question, p) for p in passages])
Higher is more relevant. In Dhara it reorders the top 50 passages from a rank fusion of BM25, a citation-graph retriever and bge-m3 dense and sparse search.
Training
- Questions: written for training-split judgments and sections only, the same way as the DharaBench benchmark, so no judgment or section in dev or test is seen in training.
- Positives: passages of the source judgment that cite the provisions behind the question, or the provision's own text for statute questions.
- Hard negatives: top results from the live retriever that are not the answer. A passage from another judgment citing the same law is never used as a negative, since it is often a right answer too; for statute questions the old/new counterpart is never one.
- Size: 1,596 questions, 9,235 pairs (about 4.8 negatives per question).
- Setup: binary cross-entropy with positives weighted by the negative ratio, 2 epochs, learning rate 2e-5, effective batch 16, 512 tokens.
The first version used negatives that included same-law passages and trained for one epoch at 320 tokens. It lost to the retrieval it was reranking; removing those negatives is what made the difference.
Evaluation
DharaBench test split: 150 questions phrased with new-code numbers whose answer is a judgment citing only the old code. Right judgment in the top 10:
| Setup | Recall@10 |
|---|---|
| BM25, question as typed | 0.27 |
| Dense only (bge-m3) | 0.31 |
| BM25 + citation graph + bge-m3, rank fusion (citation graph x2) | 0.38 |
| LLM reranker over the fusion | 0.37 |
| First fine-tuned version over the fusion | 0.36 |
| This model over the fusion | 0.43 |
On the dev split it scores 0.49, against 0.47 for the weighted fusion. With 150 questions per split, figures carry about ±0.04 of noise. It ranks better from the even fusion than from one that already weights the citation graph, and it does not improve statute ranking. Full tables: eval/results.
Dhara also uses its top score over the retrieved sections to decline questions outside criminal law. With a threshold fitted on dev to answer 95% of in-scope questions, it declines about half of the out-of-scope ones on test (civil, tax, family law).
Limits
- Trained on Supreme Court criminal judgments and the six criminal codes only; other courts and areas of law are out of its domain.
- English only.
- Questions were generated from headnotes, so they are cleaner than real queries.
- A ranking aid for research, not legal advice.