Model Card for albert-fa-sentipers-3class
A 3-class Persian sentiment classifier (negative / neutral / positive), fine-tuned from shekar-ai/albert-base-v2-persian-zwnj-naab-mlm on the DeepSentiPers dataset of Persian product review sentences.
Model Details
Model Description
This model classifies Persian-language text into three sentiment classes: negative, neutral, and positive. It was built by attaching a sequence classification head to a pretrained ALBERT-base Persian MLM checkpoint and fine-tuning it on the DeepSentiPers dataset (Persian customer review sentences from Digikala). The dataset's original 5-point polarity scale (-2 to +2) was collapsed to 3 classes: -2/-1 -> negative, 0 -> neutral, +1/+2 -> positive Training used the back-translation-augmented split (translation.csv) and was evaluated on the dataset's original held-out test.csv split.
- Developed by: MahdiAlikhah
- Model type: ALBERT-base for sequence classification, 3-class Persian sentiment analysis
- Language(s) (NLP): Persian (fa)
- License: apache-2.0 (inherited from base model — confirm against shekar-ai/albert-base-v2-persian-zwnj-naab-mlm's own license before relying on this)
- Finetuned from model: shekar-ai/albert-base-v2-persian-zwnj-naab-mlm
Model Sources
Uses
Direct Use
Sentence- or short-passage-level sentiment classification of Persian text into negative, neutral, or positive, particularly for text similar to its training domain (product reviews).
Out-of-Scope Use
Not evaluated on domains far from product reviews (e.g. social media, news, formal writing); not designed for languages other than Persian or mixed-language text; not suitable as a sole signal for high-stakes decisions without human review.
Bias, Risks, and Limitations
Training data skews positive after the 3-class collapse (positive labels outnumber negative by roughly 3-4x), partially addressed with class-weighted loss during training. The dataset is sourced from a single e-commerce platform's product reviews, so sentiment expressed in other registers or dialects may not transfer well. Collapsing the original 5-point scale to 3 classes also removes some intensity distinctions near class boundaries.
Recommendations
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Validate performance on your own domain/data before deploying outside product-review-style Persian text.
How to Get Started with the Model
Use the code below to get started with the model.
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_id = "MahdiAlikhah/albert-base-v2-persian-sentiment-sentipers"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
text = "این محصول عالی بود"
inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
logits = model(**inputs).logits
pred = logits.argmax(-1).item()
print(model.config.id2label[pred])
Training Details
Training Data
DeepSentiPers (https://github.com/JoyeBright/DeepSentiPers) — Persian sentiment corpus of Digikala product review sentences. Training file: translation.csv (back-translation-augmented, approximately 14,046 rows before splitting). Test file: test.csv (1,854 rows, original held-out split, not augmented or seen during training). Original 5-point polarity labels (-2 to +2) collapsed to 3 classes (negative/neutral/positive).
Training Procedure
Preprocessing
Text normalized with hazm's Normalizer (Persian half-space/ZWNJ normalization), then tokenized with the base model's tokenizer (max_length=128, truncation and padding to max length).
Training Hyperparameters
- Training regime: fp32
- Epochs: 5 (best checkpoint selected by validation macro-F1; epoch 4 was selected, as validation F1 peaked there before declining in epoch 5)
- Batch size: 16 (train), 32 (eval)
- Learning rate: 2e-5
- Weight decay: 0.01
- Loss: class-weighted cross-entropy (balanced class weights) to counteract training-set class imbalance
Speeds, Sizes, Times
Train runtime: approximately 26 minutes (1,574s) for 5 epochs, 3,955 steps total, on a single T4 GPU (Google Colab).
Evaluation
Testing Data, Factors & Metrics
Testing Data
DeepSentiPers test.csv (1,854 rows), 3-class-collapsed, normalized identically to training data. Never included in training or in the back-translation augmentation.
Metrics
Accuracy, and macro F1 (chosen as the primary metric because of class imbalance, so the minority negative class isn't washed out by the larger neutral/positive classes).
Results
Accuracy: 0.830. F1 (macro): 0.802. Eval loss: 0.714.
Summary
Roughly 83% accuracy and 0.80 macro-F1 on the held-out test set. For context, prior work on the full, uncollapsed 5-class DeepSentiPers task reported substantially lower macro-F1 (about 0.71 for a BERT-based model), consistent with 3-class sentiment being an easier task than 5-class.
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: NVIDIA T4 GPU
- Cloud Provider: Google Colab
Technical Specifications
Model Architecture and Objective
ALBERT-base architecture with a sequence classification head (linear layer over the pooled [CLS] output), 3-way softmax output, fine-tuned with cross-entropy loss.
Hardware
Single NVIDIA T4 GPU (Google Colab)
Software
transformers 4.57.3, tokenizers 0.22.2, PyTorch, datasets, hazm, accelerate, evaluate, scikit-learn.