Binary Product Review Sentiment with DistilBERT

Model description

This 67M parameter model is a fine-tuned version of bert-base-uncased trained on the Cleanlab Amazon Reviews dataset. It achieves the following results on the evaluation set: Loss: 0.2724 Accuracy: 0.8920

Launch it locally

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="profautorino/binary-product-review-sentiment-with-distilbert",
)

print(classifier("The headphones sound great and are comfortable to wear."))

Batch inference

For batch inference, pass a list of review strings:

reviews = [
    "This product exceeded my expectations!",
    "It broke after one day and was a waste of money.",
]
predictions = classifier(reviews, truncation=True)

for review, prediction in zip(reviews, predictions):
    print(f"{prediction['label']} ({prediction['score']:.3f}): {review}")

Intended uses & limitations

Intended to be used for batch sentiment analysis of product reviews in testing. Use in production is not advised. Use at your own risk.

Training and evaluation data

See Cleanlab/amazon-reviews for details: https://huggingface.co/datasets/Cleanlab/amazon-reviews

Training procedure

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 63
• eval_batch_size: 63
• seed: 42
• optimizer: AdamW with epsilon=2e-5
• lr_scheduler_type: linear
• num_epochs: 3

Training results

• Epoch 1 average training loss: 0.4846
• Epoch 2 average training loss: 0.3387
• Epoch 3 average training loss: 0.2579