DeBERTa-v3-xsmall-Zyda-2-quality

Model Overview

This model is a fine-tuned version of agentlans/deberta-v3-xsmall-zyda-2 designed for text quality assessment. It achieves the following results on the evaluation set:

• Loss: 0.3165
• MSE: 0.3165

Dataset Information

The model was trained on the Text Quality Meta-Analysis Dataset, which is a comprehensive collection of sentences with associated quality metrics derived from multiple sources and methods. This dataset combines text from various sources with quality scores from different models to create a thorough assessment of sentence quality.

In this context, "quality" refers to legible English sentences that are not spam and contain useful information. It does not necessarily indicate grammatical or factual correctness.

Model Description

The model is based on the DeBERTa-v3-xsmall architecture and has been fine-tuned for sequence classification tasks, specifically for assessing the quality of text inputs.

Intended Uses & Limitations

This model is intended for evaluating the quality of text inputs. It can be used for various applications such as content moderation, spam detection, or assessing the credibility of information.

Usage Example

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Load model and tokenizer
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

model_name = "agentlans/deberta-v3-xsmall-zyda-2-quality"
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=1).to(device)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Function to perform inference
def predict_score(text):
    inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True).to(device)
    with torch.no_grad():
        logits = model(**inputs).logits
    return logits.item()

# Example usage
input_text = "This product is excellent and works perfectly!"
predicted_score = predict_score(input_text)
print(f"Predicted score: {predicted_score}")

Sample Predictions

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

• Learning rate: 5e-05
• Train batch size: 64
• Eval batch size: 8
• Seed: 42
• Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
• Learning rate scheduler: Linear
• Number of epochs: 3.0

Training Results

Framework Versions

• Transformers: 4.46.3
• PyTorch: 2.5.1+cu124
• Datasets: 3.1.0
• Tokenizers: 0.20.3