cirimus/ultrabert-380M-emotions

⚠️ This model has been superseded by cirimus/modernbert-base-emotions.
💡 It is part of the collection Frontier BERT Workhorses.

Overview

This model was fine-tuned from UltraBERT-380M on the Super Emotion dataset for multi-class emotion classification. It predicts emotional states in text across seven labels: joy, sadness, anger, fear, love, neutral, surprise.

---

Model Details

• Base Model: UltraBERT-380M
• Fine-Tuning Dataset: Super Emotion
• Number of Labels: 7
• Problem Type: Multi-label classification
• Language: English
• License: Apache 2.0
• Fine-Tuning Framework: Hugging Face Transformers

---

Example Usage

Here’s how to use the model with Hugging Face Transformers:

# requires transformers>=5.2:  pip install -U "transformers>=5.2"
from transformers import pipeline

# Load the model
classifier = pipeline(
    "text-classification", 
    model="cirimus/ultrabert-380M-emotions",
    top_k=5,
    trust_remote_code=True
)

text = "I can't believe this just happened!"
predictions = classifier(text)

# Print top 3 detected emotions
sorted_preds = sorted(predictions[0], key=lambda x: x['score'], reverse=True)
top_3 = sorted_preds[:3]

print("\nTop 3 emotions detected:")
for pred in top_3:
    print(f"\t{pred['label']:10s} : {pred['score']:.3f}")

# Example output:
# Top 3 emotions detected:
#        SURPRISE   : 0.845
#        ANGER      : 0.101
#        SADNESS    : 0.057

---

How the Model Was Created

The model was fine-tuned for 2 epochs using the following hyperparameters:

• Learning Rate: 2e-5
• Batch Size: 16
• Weight Decay: 0.01
• Warmup Steps: Cosine decay scheduling
• Optimizer: AdamW
• Evaluation Metrics: Precision, Recall, F1 Score (macro), Accuracy

---

Evaluation Results

As evaluated on the joint test-set:

---

Intended Use

The model is designed for emotion classification in English-language text, particularly useful for:

• Social media sentiment analysis
• Customer feedback evaluation
• Large scale behavioral or psychological research

The model is designed for fast and accurate emotion detection but struggles with subtle expressions or indirect references to emotions (e.g., "I find myself remembering the little things you say, long after you've said them.")

---

Limitations and Biases

• Data Bias: The dataset is aggregated from multiple sources and may contain biases in annotation and class distribution.
• Underrepresented Classes: Some emotions have fewer samples, affecting their classification performance.
• Context Dependence: The model classifies individual sentences and may not perform well on multi-sentence contexts.

---

Environmental Impact

• Hardware Used: NVIDIA DGX Spark (GB10)
• Training Time: ~12 hours
• Carbon Emissions: ~0.71 kg CO2 (estimated via ML CO2 Impact Calculator)

---

Citation

If you use this model, please cite:

@misc{junque2025emotion,
  title        = {Emotion and Bias Classification with Modern Encoders},
  author       = {Junqu\'e de Fortuny, Enric},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/spaces/cirimus/emotion-detection}}
}