Qwen 2.5 0.5B - Distilled Tweet Emotion Classifier

An ultra-compact 494M parameter model fine-tuned on 4,041 Chain-of-Thought (CoT) reasoning traces distilled from GLM 5.3 Flash for Twitter emotion classification (cardiffnlp/tweet_eval).

Quick Usage

Using Apple MLX (Metal Accelerated)

mlx_lm.generate \
  --model im-shashank-bhardwaj/Qwen2.5-0.5B-Distilled-Emotion \
  --prompt "Analyze the sentiment of this tweet: 'So proud of our team today! We fought hard and won!'.
The possible labels are: anger, joy, optimism, sadness.
First, provide concise 2-3 sentence reasoning. Then on a new line output exactly: Final Label: <label>"

Using Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "im-shashank-bhardwaj/Qwen2.5-0.5B-Distilled-Emotion"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

prompt = """Analyze the sentiment of this tweet: 'So proud of our team today! We fought hard and won!'.
The possible labels are: anger, joy, optimism, sadness.
First, provide concise 2-3 sentence reasoning. Then on a new line output exactly: Final Label: <label>"""

messages = [{"role": "user", "content": prompt}]
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

---

1. Experiment Objective

This experiment aims to investigate whether the reasoning capabilities and decision boundaries of a cloud-scale large language model (GLM-4 5.3 Flash, hosted serverless via Fireworks.ai) can be effectively transferred into an ultra-compact edge model (Qwen 2.5 0.5B) on social media emotion classification (cardiffnlp/tweet_eval). Rather than standard label imitation, the core objective is to transfer step-by-step Chain-of-Thought (CoT) rationales generated across 4,041 training instances so that a sub-billion parameter model running entirely on local consumer hardware (Apple Silicon) can bridge double-digit accuracy deficits, eliminate schema parsing failures, and rival cloud LLM performance with zero inference cost and sub-second latency.

---

2. Parameter Efficiency & Training Configurations

Training was executed natively on Apple Silicon using the Apple MLX framework (mlx-lm) with Low-Rank Adaptation (LoRA) to adapt the base Qwen/Qwen2.5-0.5B-Instruct model across two parameter scale configurations:

---

3. Benchmark Results (506 Held-Out Test Samples)

Table 1: End-to-End Model Performance Comparison

---

Table 2: Per-Class F1-Score Comparison