multilingual-e5-small-aligned-quality

This model is a fine-tuned version of agentlans/multilingual-e5-small-aligned designed for assessing text quality across multiple languages.

Key Features

• Multilingual support
• Quality assessment for text
• Based on E5 small model architecture

Intended Uses & Limitations

This model is intended for:

• Assessing the quality of multilingual text
• Filtering multilingual content
• Comparative analysis of corpus text quality across different languages

Limitations:

• Performance may vary for languages not well-represented in the training data
• Should not be used as the sole criterion for quality assessment

Usage Example

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "agentlans/multilingual-e5-small-aligned-quality"

# Initialize tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

def quality(text):
    """Assess the quality of the input text."""
    inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True).to(device)
    with torch.no_grad():
        logits = model(**inputs).logits.squeeze().cpu()
    return logits.tolist()

# Example usage
score = quality("Your text here.")
print(f"Quality score: {score}")

Performance Results

The model was evaluated on a diverse set of multilingual text samples:

• 10 English text samples of varying quality were translated into Arabic, Chinese, French, Russian, and Spanish.
• The model demonstrated consistent quality assessment across different languages for the same text.

Click here for the 10 original texts and their translations.

Training Data

The model was trained on the Multilingual Parallel Sentences dataset, which includes:

• Parallel sentences in English and various other languages
• Semantic similarity scores calculated using LaBSE
• Additional quality metrics
• Sources: JW300, Europarl, TED Talks, OPUS-100, Tatoeba, Global Voices, and News Commentary

Training Procedure

Hyperparameters

• Learning rate: 5e-05
• Train batch size: 128
• Eval batch size: 8
• Seed: 42
• Optimizer: AdamW (betas=(0.9,0.999), epsilon=1e-08)
• Learning rate scheduler: Linear
• Number of epochs: 3.0

Training Results

Framework Versions

• Transformers: 4.46.3
• PyTorch: 2.5.1+cu124
• Datasets: 3.1.0
• Tokenizers: 0.20.3