multilingual-e5-small-aligned-readability

This model is a fine-tuned version of agentlans/multilingual-e5-small-aligned designed for assessing text readability across multiple languages.

Key Features

• Multilingual support
• Readability assessment for text
• Based on E5 small model architecture

Intended Uses & Limitations

This model is intended for:

• Assessing the readability of multilingual text
• Filtering multilingual content
• Comparative analysis of corpus text readability across different languages

Limitations:

• Performance may vary for languages not well-represented in the training data
• Should not be used as the sole criterion for readability assessment

Usage Example

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
import numpy as np

model_name = "agentlans/multilingual-e5-small-aligned-readability"

# Initialize tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

def readability(text):
    """Assess the readability of the input text."""
    inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True).to(device)
    with torch.no_grad():
        logits = model(**inputs).logits.squeeze().cpu()
    return logits.tolist()

# Grade level conversion function
# Input: readability value
# Output: the equivalent U.S. education grade level
def grade_level(y):
    lambda_, mean, sd = 0.8766912, 7.908629, 3.339119
    y_unstd = (-y) * sd + mean
    return np.power((y_unstd * lambda_ + 1), (1 / lambda_))

# Example
input_text = "The mitochondria is the powerhouse of the cell."
readability_score = readability(input_text)
grade = grade_level(readability_score)
print(f"Predicted score: {readability_score:.2f}\nGrade: {grade:.1f}")

Performance Results

The model was evaluated on a diverse set of multilingual text samples:

• 10 English text samples of varying readability were translated into Arabic, Chinese, French, Russian, and Spanish.
• The model demonstrated consistent readability assessment across different languages for the same text.

Click here for the 10 original texts and their translations.

Training Data

The model was trained on the Multilingual Parallel Sentences dataset, which includes:

• Parallel sentences in English and various other languages
• Semantic similarity scores calculated using LaBSE
• Additional readability metrics
• Sources: JW300, Europarl, TED Talks, OPUS-100, Tatoeba, Global Voices, and News Commentary

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 128
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3.0

Training results

Framework versions

• Transformers 4.46.3
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3