Loading…
Loading…
A 118M-parameter reranker for Arabic, English and cross-lingual search: give it a query and candidate passages from any first-stage retriever and it re-orders them.
Fine-tuned from cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 on mined hard negatives.
Code, benchmark and full results: github.com/mahmoudzahran01/arabic-english-retrieval-eval.
Companion embedder: mzahran1/arabic-english-e5-small-retrieval.
Test MRR@10 x 100 after reranking the first stage's top-50 (6,823 test queries from TyDi-QA, ARCD and XQuAD; single training run).
| First stage -> reranker | ar->ar | en->en | ar->en | en->ar | macro |
|---|---|---|---|---|---|
| multilingual-e5-small, no reranker | 76.8 | 87.8 | 41.4 | 61.7 | 66.9 |
| multilingual-e5-small -> base mMiniLM reranker | 85.3 | 90.9 | 65.7 | 80.0 | 80.5 |
| multilingual-e5-small -> this reranker | 87.2 | 91.0 | 68.9 | 83.8 | 82.7 |
| fine-tuned e5-small -> this reranker | 86.7 | 90.6 | 72.3 | 83.7 | 83.3 |
| language-gated RRF(bge-m3, BM25) -> this reranker | 89.0 | 91.3 | 75.3 | 84.4 | 85.0 |
Fine-tuning adds +3.1 [+1.8, +4.5] on ar->en and +3.8 [+2.6, +5.1] on en->ar over the base reranker (paired bootstrap 95% CI). Median latency: 94 ms to score 50 candidates on one GPU (39 ms for 20).
from sentence_transformers import CrossEncoder
reranker = CrossEncoder("mzahran1/arabic-english-reranker-mminilm")
query = "متى تأسست جامعة الملك عبدالله للعلوم والتقنية؟"
passages = [
"The King Abdullah University of Science and Technology was founded in 2009 in Thuwal.",
"The University of Maryland was founded in 1856 and is located in College Park.",
]
scores = reranker.predict([(query, p) for p in passages]) # higher = more relevant; no prefixes needed
Pointwise binary cross-entropy, positives up-weighted x4, 1 gold + 4 hard negatives per query for 20,918 training queries (including LLM-translated cross-lingual ones), 1 epoch (3,268 steps, batch 32, lr 2e-5, max 256 tokens). Negatives were mined by the base multilingual-e5-small (ranks 1-30, 0.02 margin), the distribution a reranker sees in production. Mined negatives matter: the same recipe with random negatives scored 80.1 macro and lost 5 points on English.
Apache-2.0, like the base model. The training data keeps its own licences (TyDi-QA Apache-2.0, ARCD MIT, SQuAD CC BY-SA 4.0).
cross-encoder/mmarco-mMiniLMv2-L12-H384-v1