DistilBERT (distilbert-base-uncased-finetuned-sst-2-english)
fine-tuned for positive vs negative sentiment of Amazon product reviews (Appliances + Beauty).
Label 1 = positive (4-5 stars), label 0 = negative (1-2 stars). Part of the
product-review-intelligence project.
| Model | Macro-F1 | ROC-AUC |
|---|---|---|
| This model, threshold 0.5 | 0.896 | 0.986 |
| This model, threshold 0.06 (recommended) | 0.920 | 0.986 |
| DistilBERT SST-2 zero-shot | 0.758 | 0.954 |
| TF-IDF + logistic regression (78k training reviews) | 0.919 | 0.988 |
Use the threshold 0.06 on P(positive). Fine-tuning on a balanced sample shifts the model toward "negative" relative to the natural population (82% of labelled reviews are positive); the threshold was chosen on a validation subset to maximise macro-F1.
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tok = AutoTokenizer.from_pretrained("WaleedIlyas99/distilbert-amazon-review-sentiment")
model = AutoModelForSequenceClassification.from_pretrained("WaleedIlyas99/distilbert-amazon-review-sentiment").eval()
enc = tok(["Works great but the lid cracked after two weeks."], return_tensors="pt", truncation=True, max_length=128)
p_pos = torch.softmax(model(**enc).logits, dim=-1)[0, 1].item()
print("positive" if p_pos >= 0.06 else "negative", round(p_pos, 3))
distilbert/distilbert-base-uncased-finetuned-sst-2-english