uz-sentiment-xlmr
Binary sentiment classifier for Uzbek (Latin script) product reviews, fine-tuned from xlm-roberta-base.
- Labels:
0= negative,1= positive - Input: short, informal Uzbek text (trained with
max_length=128)
Data
behbudiy/uzbek-sentiment-analysis (10k rows, e-commerce reviews).
Preprocessing: lowercasing, apostrophe normalization, whitespace cleanup; duplicate texts and texts with conflicting labels were removed. Stratified split 80/10/10 (seed 42); test set = 827 examples (507 negative / 320 positive).
Training
PyTorch + Hugging Face Trainer, 3 epochs, batch size 32, lr 2e-5, weight decay 0.01, 10% warmup, fp16 on a Colab T4 GPU (~9m45s; the same run on CPU took ~4h12m). Best checkpoint selected by validation macro-F1.
Results (test set)
| Model | Accuracy | Macro-F1 |
|---|---|---|
| TF-IDF (char n-gram 2-5) + Logistic Regression | 0.904 | 0.897 |
| XLM-R fine-tune (this model) | TODO_ACC | TODO_F1 |
The difference between the two models is not statistically significant (bootstrap 95% CI for the macro-F1 difference: [-0.014, 0.035]).
Limitations
- Labels in the dataset are noisy; many reviews are mixed-sentiment (e.g. "yaxshi, lekin ...") and labeled inconsistently.
- Most common errors: negated negatives ("yomon emas", "yomonmas") predicted as negative; slang and Russian/English code-switching; spelling variants ("yaxwi").
- Recall on the positive class is lower (~0.86) than on the negative class (~0.94).
- Trained on product reviews only; performance on other domains (news, social media) is untested.
Usage
from transformers import pipeline
clf = pipeline("text-classification", model="yusuf0902/uz-sentiment-xlmr")
print(clf("yaxshi mahsulot ekan, tavsiya qilaman"))