Hinglish Retail Intent Classifier
Fine-tuned version of google/muril-base-cased for classifying Indian ecommerce customer support messages (Hinglish) into 13 intent categories.
Performance
- Test accuracy: 97.6%
- Macro F1: 97.6%
| Label | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| cancel_order | 1.00 | 1.00 | 1.00 | 22 |
| damaged_item | 1.00 | 1.00 | 1.00 | 23 |
| delivery_complaint | 0.83 | 0.87 | 0.85 | 23 |
| discount_query | 1.00 | 1.00 | 1.00 | 23 |
| exchange_product | 1.00 | 1.00 | 1.00 | 21 |
| other | 1.00 | 1.00 | 1.00 | 19 |
| payment_issue | 1.00 | 1.00 | 1.00 | 23 |
| product_availability | 1.00 | 1.00 | 1.00 | 23 |
| product_information | 1.00 | 1.00 | 1.00 | 21 |
| refund_status | 1.00 | 1.00 | 1.00 | 23 |
| return_request | 1.00 | 1.00 | 1.00 | 23 |
| track_order | 0.95 | 0.82 | 0.88 | 22 |
| wrong_item | 0.91 | 1.00 | 0.95 | 21 |
Usage
from transformers import pipeline
classifier = pipeline(
"text-classification",
model="Hari5115/hinglish-retail-intent-classifier"
)
result = classifier("mera order kab aayega?")
print(result)
# [{'label': 'track_order', 'score': 0.96}]
Intents supported
| Label | Description |
|---|---|
track_order |
Customer asking where their order is or when it arrives |
cancel_order |
Customer wants to cancel a placed order |
exchange_product |
Customer received wrong size/colour, wants a swap |
refund_status |
Customer asking about refund timeline or whether it was processed |
delivery_complaint |
Late delivery, not delivered, delivery agent issue |
damaged_item |
Product arrived broken, damaged, or defective |
wrong_item |
Completely wrong product was delivered |
return_request |
Customer wants to return a product |
payment_issue |
Money deducted but order not placed, double charge |
product_availability |
Asking if an item is in stock or available in a size/colour |
product_information |
Asking about product details, material, dimensions |
discount_query |
Promo code not working, coupon issues, asking for discount |
other |
Anything that does not fit the above |
Training data
Trained on Hari5115/hinglish-retail-intent-dataset
Training config
- Base model:
google/muril-base-cased - Epochs: 5
- Learning rate: 2e-5
- Batch size: 16 (train), 32 (eval)
- Max sequence length: 128
- Best model selected by macro F1 on validation set
Limitations
- Synthetic data — may not capture all real-world linguistic variation
- Primarily Hindi-English mixing; does not cover Tamil-English, Telugu-English, or other Indian language combinations
- Skewed toward common intents; rare edge cases are underrepresented