paraphrase-albert-small-v2 fine-tuned for emotion classification

A small ALBERT-based Classifier that assigns one of six Emotions to a short English text: sadness, joy, love, anger, fear or surprise. It is sentence-transformers/paraphrase-albert-small-v2 fine-tuned on dair-ai/emotion.

Code, training notebook and API: https://github.com/PierreBoissier/emotion_classification

Usage

from transformers import pipeline

classifier = pipeline("text-classification", model="pboissie/paraphrase-albert-small-v2-emotion", top_k=None)
classifier("i am so happy to see you again today")

Texts longer than 256 tokens are truncated.

Results

Evaluated on the 2,000-text test Split of dair-ai/emotion:

The weakest Emotion is surprise (recall 0.636), mostly mistaken for fear. Love is often mistaken for joy.

Training

• Data: the dair-ai/emotion train Split (16,000 texts) for training; the validation Split (2,000 texts) is only used to monitor accuracy, not to train or select weights.
• 10 epochs, learning rate 2e-5, weight decay 0.01, batch size 32 with 4 gradient accumulation steps, AdamW with a linear schedule (5% warmup), mixed precision, on a Colab GPU.
• The weights of the last epoch are kept. Validation accuracy reaches about 0.93 and is flat over the last epochs.

Limitations

• English only, and trained on short texts: the dataset is made of English tweets labelled from hashtags, so labels are noisy and the Emotions reflect that style of writing.
• Classes are imbalanced (surprise and love are rare), which explains their lower scores.
• Each text gets exactly one Emotion; mixed or neutral feelings are not modelled.
• Not intended for decisions about people: it is a portfolio project, not a validated tool.