Mizan-Rerank-v3-Turbo

The accuracy of Mizan-Rerank-v3 at half the size and twice the speed: a compact Arabic reranker that tells the passage that answers the question apart from passages that only look like they do.

Try it: live demo — rerank your own passages, run the eleven adversarial trap cases, and switch between this model and Mizan-Rerank-v3 to compare.

The Mizan reranker family

Model	Parameters	Throughput	Held-out mean nDCG@10	Fiqh nDCG@10
Mizan-Rerank-v3-Turbo (this model)	150M	2,487 pairs/s	0.871	0.748
Mizan-Rerank-v3	306M	1,238 pairs/s	0.826	0.669

Pick Turbo for the best accuracy per millisecond, v3 if you prefer the larger model with the same behaviour as the previous release.

Held-out mean nDCG@10 is the mean over the four held-out sets both cards share (NamaaMrTydi unseen, short adversarial, long-context adversarial, Multi-LLM). Mizan-Rerank-v3's own card quotes 0.845, which averages a fifth set (Arabic Hard Negatives unseen) that cannot be used here: only 10 of its 12,373 cases are unseen for Turbo.

Mizan-Rerank-v3-Turbo is a 150M-parameter Arabic cross-encoder. It starts from the Mizan-Embeddings-0.1B encoder and is trained in three stages: broad Arabic relevance with hard negatives and teacher scores, in-domain distillation on production retrieval candidates, then v3's adversarial and long-context recipe. It learns to separate the correct passage from adversarial "trap" passages that share most of its words but change its meaning: a negated ruling, a swapped entity, a shifted number or date, an exception applied to the wrong case.

The result is the most accurate model in the family while being the smallest: half the parameters of Mizan-Rerank-v3, twice its throughput, 3.2× less peak GPU memory, and higher scores on every held-out benchmark we measure, including the real production fiqh queries.

Highlights

• Best accuracy in the family. Held-out mean nDCG@10 is 0.871 at 150M parameters: Mizan-Rerank-v3 scores 0.826, bge-reranker-v2-m3 (568M) 0.777, gte-multilingual-reranker-base 0.752 and Mizan-Rerank-V2 0.685.
• Fast and small. On one RTX 4090 in fp16 it scores 2,487 pairs/s (2.0× v3, 4.1× bge-reranker-v2-m3) with 0.34 GiB peak GPU memory (see Efficiency).
• Best on real production queries. On 224 real fiqh search queries with the production retriever's candidate lists, nDCG@10 is 0.748 against 0.669 for v3 and 0.737 for bge-reranker-v2-m3 (+0.078 over v3, 95% CI [+0.057, +0.101]). It keeps 0.735 of the relevant rulings in production's top-8 filter (v3 0.676).
• Rarely puts a contradiction on top. Across 2,666 held-out adversarial queries, the top-ranked passage is a contradicting trap 19.3% of the time. For Mizan-Rerank-v3 the figure is 32.5%, for bge-reranker-v2-m3 56.8% and for Mizan-Rerank-V2 73.0%.
• Scores on v3's scale, without calibration. Answers score high, and traps, partially relevant and off-topic passages score low, on fiqh and general text alike, so a fixed cut-off works across domains (see Practical notes).

Usage

Sentence Transformers

pip install -U sentence-transformers

from sentence_transformers import CrossEncoder

model = CrossEncoder("ALJIACHI/Mizan-Rerank-v3-Turbo", max_length=3072)

query = "ما هي فوائد فيتامين د؟"
passages = [
    "يساعد فيتامين د على امتصاص الكالسيوم وتقوية العظام، كما يدعم عمل الجهاز المناعي.",
    "يستخدم فيتامين د في بعض الصناعات الغذائية كمادة مضافة لتدعيم الحليب.",
    "أطلقت وزارة الزراعة حملة وطنية لزيادة الوعي بأهمية الزراعة العضوية.",
]

scores = model.predict([(query, passage) for passage in passages])
print(scores)  # ≈ [0.62, 0.08, 0.0002]  (sigmoid relevance scores; the second passage is on-topic but does not
#               answer the question, so it stays below the 0.1 cut-off while the answer is ranked first)

for hit in model.rank(query, passages, return_documents=True):
    print(f"{hit['score']:.3f}  {hit['text']}")

Transformers

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("ALJIACHI/Mizan-Rerank-v3-Turbo")
model = AutoModelForSequenceClassification.from_pretrained(
    "ALJIACHI/Mizan-Rerank-v3-Turbo", torch_dtype=torch.float16
).to("cuda").eval()

query = "ما هي فوائد فيتامين د؟"
passages = [
    "يساعد فيتامين د على امتصاص الكالسيوم وتقوية العظام، كما يدعم عمل الجهاز المناعي.",
    "أطلقت وزارة الزراعة حملة وطنية لزيادة الوعي بأهمية الزراعة العضوية.",
]

with torch.inference_mode():
    features = tokenizer([query] * len(passages), passages, padding=True, truncation=True,
                         max_length=3072, return_tensors="pt").to("cuda")
    scores = torch.sigmoid(model(**features).logits.squeeze(-1).float())

for passage, score in sorted(zip(passages, scores.tolist()), key=lambda item: -item[1]):
    print(f"{score:.3f}  {passage}")

Practical notes

• No trust_remote_code needed. The architecture is plain ModernBERT, so the model loads with any Transformers version that supports ModernBertForSequenceClassification (4.48+). Mizan-Rerank-v3 ships custom code and needs trust_remote_code=True.
• Sequence length. The model was trained on query + passage inputs of up to 3,072 tokens and is benchmarked here at 2,048. The architecture accepts 8,192 positions, but quality beyond 3,072 tokens has not been measured. If your query + passage inputs fit in 512 tokens, max_length=512 is much faster (it is what the fiqh evaluation and the speed benchmark use).
• Scores. The sigmoid of the single output logit is a relevance score on Mizan-Rerank-v3's scale, so coarse cut-offs work (production uses sigmoid > 0.1 to keep the top 8), but it is not a probability of correctness: tune any threshold on your own data.
• Small and fast. fp16 throughput is 2,487 pairs/s at 0.34 GiB, so it can rerank about twice as many candidates as Mizan-Rerank-v3 in the same latency budget.

Evaluation

All five models were scored by the same script with the same inputs: float32, max_length=2048, and the full candidate list of every query. Metrics use graded gains (correct passage 3, partially correct passage 1, trap or irrelevant passage 0).

Held-out benchmarks

¹ Queries whose question and correct passage never occur in the Turbo training pool (checked by exact query/passage hash match against the full pool). ² Internal test sets, not released. They share no queries or passages with the training data (verified by exact match), and the long-context split is by source article.

The averages cover the four held-out sets above. Arabic Hard Negatives is reported separately and never averaged: 12,363 of its 12,373 queries occur in the Turbo training pool (Mr. TyDi replay and Arabic MS MARCO-style triplets), so only 10 cases are unseen and a score on it would not measure generalisation. Its full-set nDCG@10 is 0.9225 for Turbo, 0.9103 for v3 and 0.9082 for bge-reranker-v2-m3, which is why it is excluded here.

Where the gain comes from: ranking the correct passage first. Hit@1 is where Turbo differs most from the other models, especially when traps share most of their text with the answer:

Real production queries (fiqh)

224 real fatwa-search queries from the production trace database, each with the production retriever's top-25 candidates and graded judgments (0-3) from two LLM judges (weighted kappa 0.81). Scoring matches production: fp16, max_length=512, sigmoid, keep the top 8 above 0.1.

misleading@3 counts queries with a ruling that answers a different case in the top 3 (lower is better). relevant_kept@8 is the share of relevant rulings that survive production's top-8 + sigmoid > 0.1 filter.

Contradiction ranked first

For RAG the worst failure is not a missing answer but a passage that says the opposite ranked at the top: a negated ruling, a reversed condition, a wrong number. Every adversarial test query has one correct passage and several contradicting traps. The table shows how often a trap was ranked #1 (lower is better):

Across all 2,666 queries, Turbo ranks a trap first 19.3% of the time. For Mizan-Rerank-v3 the figure is 32.5%; for bge-reranker-v2-m3, 56.8%; for gte base, 63.8%; for Mizan-Rerank-V2, 73.0%:

Reproduce with python compare_generated_test_models.py --dataset-file <test.jsonl> ... --chart contradiction_first.png (in the training repository; the test sets are internal).

Full per-benchmark tables (nDCG@10 / MRR@10 / Hit@1), including full-set public scores

nDCG@10

MRR@10

Hit@1

³ 12,363 of these queries occur in the Turbo training pool (and 97% in v3's), so the full-set score is not a fair measure for either Mizan model and is not bolded or used in any average. The unseen subset is only 10 cases.

Efficiency

Every model scored the same 2,048 Arabic query–passage pairs from MTEB NamaaMrTydi. The run used one NVIDIA RTX 4090, float16, batch size 32 and max_length=512, after a warm-up:

At the same latency budget, Turbo can rerank twice as many candidates as Mizan-Rerank-v3 and four times as many as bge-reranker-v2-m3, at the highest held-out accuracy of the five.

Framework versions

Python 3.10.14 · PyTorch 2.8.0+cu126 · Transformers 4.55.4 · Sentence Transformers 5.4.1 · Accelerate 1.10.0 · Tokenizers 0.21.0

Citation

@software{Mizan_Rerank_v3_Turbo_2026,
  author    = {Ali Aljiachi},
  title     = {Mizan-Rerank-v3-Turbo: Compact Arabic Long-Context Reranker},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/ALJIACHI/Mizan-Rerank-v3-Turbo}
}

License

Apache 2.0.