RaahNaame-Koochik-v1 (راه‌نامه)

RaahNaame-Koochik-v1 is a 308M-parameter Persian text-embedding model. It is google/embeddinggemma-300m, further trained by distilling two much larger teachers into it:

• F2LLM-v2-14B for classification, clustering, pair-classification, STS and general-text structure.
• Gemini Embedding 2 for retrieval (query → passage ranking).

On MTEB(fas, v2) (all 52 tasks) it scores 71.46 Mean(Task) / 73.90 Mean(TaskType). That is +3.7 points over the EmbeddingGemma-300m base and ahead of every model up to 1.7B parameters on the benchmark.

Results: MTEB(fas, v2), 52 tasks

*Borda rank among the 32 models with results on all 52 tasks in the public embeddings-benchmark/results repository, computed by us (Nov 2026). The official leaderboard may filter models differently.

By task type: Classification 76.33 · Clustering 69.37 · Pair classification 85.31 · Reranking 70.50 · Retrieval 60.06 · STS 78.77 · Bitext mining 76.96.

Evaluated with mteb 2.27.1 through its reference implementation for this model (mteb.get_model("Reza2kn/RaahNaame-Koochik-v1"), fp32), using the model's own prompts (below).

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Reza2kn/RaahNaame-Koochik-v1")

queries = ["پایتخت ایران کجاست؟"]
docs = ["تهران پایتخت و بزرگ‌ترین شهر ایران است.", "اصفهان یکی از شهرهای تاریخی ایران است."]

q = model.encode_query(queries)      # uses "task: search result | query: "
d = model.encode_document(docs)      # uses "title: none | text: "
print(model.similarity(q, d))

For other tasks, pass the matching prompt name: Classification, Clustering, STS, PairClassification, Summarization. For example, model.encode(texts, prompt_name="Classification"). The prompts are the EmbeddingGemma ones:

Output is a 768-d L2-normalized vector; use cosine similarity. Matryoshka truncation (512/256/128) is inherited from the base model but was not re-tuned.

Training

• Student: google/embeddinggemma-300m (full model including the Dense heads).

• Losses: in-batch InfoNCE (w 0.25) plus a KL between student and teacher ranking distributions over positives, hard negatives and in-batch candidates (w 0.75). On general-text batches it adds relational (similarity-matrix) KL and projected cosine to the 14B teacher's 5120-d space.

• Retrieval teacher: Gemini Embedding 2. Its similarity logits (τ = 0.02, calibrated to the 14B's confidence) are the targets on retrieval batches. Hard negatives were mined with Gemini.

• Other teacher: F2LLM-v2-14B (τ = 0.07).

• Schedule: two stages.

  1. 5k steps on teacher-labelled Persian data: Yalda-derived query–passage pairs, F2LLM task data, a Persian news/medical/science/social domain set and general web text.
  2. 5k steps adding ~285k supervised groups built from the train splits of Persian benchmark datasets, with 14B targets.

  LR 5e-6, cosine schedule.

• Final model: uniform average of stage-2 checkpoints at steps 3000, 4000 and 5000, then weight-merged 75/25 with the stage-1 model. This keeps the classification gains and recovers retrieval.

Data hygiene

Every text from every evaluation split of all 52 MTEB(fas, v2) tasks (167k texts), plus 33.5k benchmark test queries, was removed from all training data before training. No test or evaluation data was used. Train splits of the following MTEB tasks were used, so the model is not zero-shot on them:

SynPerQAPC, SynPerQARetrieval, PersianFoodSentimentClassification, SynPerTextToneClassification.v3, SynPerTextKeywordsPC, SAMSumFa, SynPerSTS, MassiveIntentClassification, MassiveScenarioClassification, SIDClassification.v2, SIDClustring, SynPerChatbotSumSRetrieval, SynPerChatbotSatisfactionLevelClassification, DigikalamagClassification, DigikalamagClustering, PerShopDomainClassification, PerShopIntentClassification, DeepSentiPers.v2, PersianTextEmotion.v2, SynPerChatbotConvSAToneUserClassification, SynPerChatbotConvSAToneChatbotClassification, NLPTwitterAnalysisClassification.v2, NLPTwitterAnalysisClustering, SynPerChatbotRAGFAQRetrieval, SynPerChatbotRAGSumSRetrieval, SynPerChatbotRAGFAQPC, FarsiParaphraseDetection, StyleClassification

Retrieval train pairs (test queries removed) came from the Persian versions of MSMARCO, HotpotQA, NQ, FiQA, ArguAna, SciFact, ClimateFEVER, NFCorpus, CQADupStack and PersianWebDocumentRetrieval, plus PQuAD/PersianQuAD.

License

This model is a derivative of EmbeddingGemma and is distributed under the Gemma Terms of Use. By using it you agree to those terms and the Gemma Prohibited Use Policy.

Citation

@misc{raahnaame2026,
  title  = {RaahNaame: Distilling a 14B embedding model and Gemini into EmbeddingGemma for Persian},
  author = {Reza Sayar},
  year   = {2026},
  url    = {https://huggingface.co/Reza2kn/RaahNaame-Koochik-v1}
}