SPLADE-Amharic-Base

This is a SPLADE Sparse Encoder model finetuned from rasyosef/roberta-base-amharic using the sentence-transformers library. It maps sentences & paragraphs to a 32000-dimensional sparse vector space and can be used for semantic search and sparse retrieval in Amharic.

This model was presented in the paper The Multilingual Curse at the Retrieval Layer: Evidence from Amharic.

Official code repository: https://github.com/rasyosef/amharic-neural-ir

Model Details

Model Description

• Model Type: SPLADE Sparse Encoder
• Base model: rasyosef/roberta-base-amharic
• Maximum Sequence Length: 510 tokens
• Output Dimensionality: 32000 dimensions
• Similarity Function: Dot Product
• Language: am
• License: mit

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Sparse Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sparse Encoders on Hugging Face

Full Model Architecture

SparseEncoder(
  (0): MLMTransformer({'max_seq_length': 510, 'do_lower_case': False, 'architecture': 'XLMRobertaForMaskedLM'})
  (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 32000})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SparseEncoder

# Download from the 🤗 Hub
model = SparseEncoder("rasyosef/splade-amharic-base")
# Run inference
sentences = [
    'ለውጭ ገበያ በሚቀርበው የኢትዮጵያ ቡና ላይ የተጋረጠው ፈተና',
    'የኢትዮጵያ ዋነኛ የውጭ ምንዛሬ ምንጭ የሆነው ወደ ውጭ የሚላክ ቡና ዘርፍ በአሁኑ ጊዜ ከፍተኛ ውጥረት ውስጥ ገብቷል። በዚህ የተነሳም የኢትዮጵያ ቡናና ሻይ ባለሥልጣንን ጨምሮ የሚመላካታቸው ሁሉ ቡና ላኪዎችና አምራቾች ያከማቹትን ቡና በፍጥነት ወደ ዓለም ገበያ እንዲያወጡ ጥሪ እያቀረቡ ነው ።',
    'የቻይናው ፕሬዝዳንት ዚ ጂንፒንግ ከትራምፕ ጋር ባደረጉት ጉባኤ ትኩረታቸው በሁለቱ ሀገራት መካከል ለወራት ከተፈጠረ ውጥረት እና የንግድ ጦርነት በኋላ የተረገጋጋ ግንኙነትን ማስቀጠል ነበር። ከፑቲን ጋር ደግሞ ዢ ለሁለቱ አገራት ስልታዊም ሆነ ኢኮኖሚያዊ ጠቀሜታ ረጅም ጊዜ የዘለቀውን አጋርነትን ይበልጥ ማጠናከር ላይ ነበር ትኩረታቸው።',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 32000]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[45.2024, 19.3316,  0.0000],
#         [19.3316, 48.7685,  8.5323],
#         [ 0.0000,  8.5323, 63.2857]])

Evaluation

Metrics

Sparse Information Retrieval

• Evaluated with SparseInformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 245,876 training samples
• Columns: anchor, positive, and negative
• Loss: SpladeLoss with these parameters:

  {
      "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
      "document_regularizer_weight": 0.001,
      "query_regularizer_weight": 0.002
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• gradient_accumulation_steps: 2
• learning_rate: 6e-05
• num_train_epochs: 6
• lr_scheduler_type: cosine
• warmup_ratio: 0.025
• fp16: True
• optim: adamw_torch_fused
• batch_sampler: no_duplicates

Citation

@inproceedings{alemneh2026amharicir,
  title     = {The Multilingual Curse at the Retrieval Layer: Evidence from Amharic},
  author    = {Alemneh, Yosef Worku and Mekonnen, Kidist Amde and de Rijke, Maarten},
  booktitle = {Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM), ACL 2026},
  year      = {2026},
}