Hebrew QA Retriever (fine-tuned multilingual-e5-base)
Retrieves the Hebrew paragraph that answers a Hebrew question. Fine-tuned from intfloat/multilingual-e5-base on the
answerable questions of HeQ, with mined hard negatives
and CachedMultipleNegativesRankingLoss (batch 128, 2 epochs, lr 2e-05).
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("ElyKok/hebrew-qa-retriever-e5-base")
q = model.encode("query: מי ניסח את מגילת העצמאות?", normalize_embeddings=True)
p = model.encode(["passage: ..."], normalize_embeddings=True)
Use the query: and passage: prefixes, as with the base model.
Results (HeQ test set, 1,072 questions, search over all 4,930 paragraphs)
| Model | R@1 | R@5 | R@10 | MRR@10 | NDCG@10 | R@5 Wikipedia | R@5 Geektime |
|---|---|---|---|---|---|---|---|
| BM25 (keyword search) | 73.1 | 86.8 | 89.8 | 79 | 81.7 | 91.2 | 82.2 |
| multilingual-e5-base | 67.4 | 81.7 | 86.2 | 73.4 | 76.5 | 86.2 | 77 |
| multilingual-e5-large | 72.7 | 87.8 | 91.4 | 79.2 | 82.2 | 90.8 | 84.6 |
| Qwen3-Embedding-0.6B | 55.4 | 70.6 | 76.3 | 62.2 | 65.6 | 74.3 | 66.8 |
| Fine-tuned e5-base (ours) | 67.3 | 82.8 | 86.5 | 74.1 | 77.1 | 87 | 78.6 |
Test articles are disjoint from training articles. HeQ is licensed CC BY 4.0 (Webiks / MAFAT). Limitation: HeQ questions were written while looking at the paragraph, so they share more wording with it than real user queries would.