multilingual-e5-small-aligned-sentiment

This model is a fine-tuned version of agentlans/multilingual-e5-small-aligned designed for assessing text sentiment across multiple languages.

Key Features

• Multilingual support
• Sentiment assessment for text
• Based on E5 small model architecture

Intended Uses & Limitations

This model is intended for:

• Assessing the sentiment of multilingual text
• Filtering multilingual content
• Comparative analysis of corpus text sentiment across different languages

Limitations:

• Performance may vary for languages not well-represented in the training data
• Should not be used as the sole criterion for sentiment assessment

Usage Example

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "agentlans/multilingual-e5-small-aligned-sentiment"

# Initialize tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

def sentiment(text):
    """Assess the sentiment of the input text."""
    inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True).to(device)
    with torch.no_grad():
        logits = model(**inputs).logits.squeeze().cpu()
    return logits.tolist()

# Example usage
score = sentiment("Your text here.")
print(f"Sentiment score: {score}")

Performance Results

The model was evaluated on a diverse set of multilingual text samples:

• 10 English text samples of varying sentiment were translated into Arabic, Chinese, French, Russian, and Spanish.
• The model demonstrated consistent sentiment assessment across different languages for the same text.

Click here for the 10 original texts and their translations.

Training Data

The model was trained on the Multilingual Parallel Sentences dataset, which includes:

• Parallel sentences in English and various other languages
• Semantic similarity scores calculated using LaBSE
• Additional sentiment metrics
• Sources: JW300, Europarl, TED Talks, OPUS-100, Tatoeba, Global Voices, and News Commentary

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 128
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3.0

Training results

Framework versions

• Transformers 4.46.3
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3