e5-base-german-sentence-similarity

This model is a fine-tuned version of intfloat/e5-base on german subset of the stsb_multi_mt dataset. It achieves the following results on the evaluation set:

Validation:

• Loss: 0.9118
• Pearson: 0.7952 Test:
• Loss: 1.2162
• Pearson: 0.7252

Usage

#install sentence-transformers
!pip install -U sentence-transformers

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("kaixkhazaki/e5-base-german-sentence-similarity")

sentences = [
    'Ein älterer Herr genießt die Natur auf einer Parkbank.',
    'Ein alter Mann vertieft sich in eine Zeitung im Stadtpark.',
    'Ein Teenager hört Musik auf einer Bank.'
]
embeddings = model.encode(sentences)

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)

>>
tensor([[1.0000, 0.9044, 0.7919],
        [0.9044, 1.0000, 0.7757],
        [0.7919, 0.7757, 1.0000]])

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

• Dataset: STS Benchmark

Training procedure

additional dropout implemented on model to avoid overfitting(hidden_dropout_prob=0.25 ,attention_probs_dropout_prob = 0.25)

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 500
• num_epochs: 12

Training results

Framework versions

• Transformers 4.48.0
• Pytorch 2.2.2
• Datasets 3.2.0
• Tokenizers 0.21.0