Superseded: this model is from an earlier Jeff release and is no longer updated. The current release is Jeff v1.3: mstrasser/jeff-adapter-emotion. This v1.2 adapter loads only on the v1.2 base.

Jeff-Qwen3.5-0.8B-emotion

Emotion in short comments: picks the strongest of 27 emotions, or neutral, in a short comment or message.

Give it a short comment or message and it returns a probability for each of 27 emotions and neutral, using the GoEmotions labels (admiration, amusement, anger and so on). Many comments carry more than one emotion, so read the result as a spread, not a single verdict. It is a LoRA adapter for Jeff-Qwen3.5-0.8B v1.2, a small open decision model: it answers in one forward pass, with no generated text to parse.

Results

Test set: GoEmotions test split, 5,408 rows. The official GoEmotions test split (simplified configuration); never trained on. It is included in this repo as test.jsonl.

Calibration error is the expected calibration error over 15 confidence bins, after each model's fitted temperature: the average gap between how confident the model is and how often it is right. Lower is better. The untrained Qwen3.5-0.8B is read out from its own scores for the option letters.

Against Qwen3.8-27B

Here Jeff + emotion answers first. Below a confidence threshold, the query is also sent to Qwen3.8-27B and the 27B's answer is used. Each adapter's threshold is the fastest one that still beats the 27B alone by at least 1 point of accuracy on the adapter's calibration rows (not the test rows).

The threshold is 0.00: on the calibration rows the adapter alone already beat the 27B by more than a point, so no query is passed on. The route is 42.5× faster than the 27B alone.

How this was measured: a fixed random sample of 500 held-out test rows, the same rows for both routes, on an Apple M4 Max (128 GB) with both models in MLX, one at a time. Qwen3.8-27B ran with 8-bit weights, prompted, with step-by-step reasoning off. Times are the mean per query over the whole sample, from prompt to answer. This is a 500-row sample, not the full test set.

Speed and memory

One request at a time on an NVIDIA RTX PRO 6000: 675 requests per setting, 75 test prompts of each of the nine adapters, interleaved. Measured for the adapters as a group, not for each adapter, so treat these as typical figures:

Jeff's speed and memory on the Mac: not measured yet.

Where it goes wrong

On the full test set:

• This is a hard task, and the adapter is right only about 6 times in 10. The main problem is neutral: mild emotions such as approval, annoyance, disapproval and realization are most often read as neutral, and neutral comments are sometimes read as curiosity or admiration. Clear emotions such as gratitude, amusement and love are read well.
• GoEmotions comments often have more than one gold emotion; the probabilities are more useful than the top answer.

Commonest confusions (right answer → chosen instead, rows):

Weakest answers (answers with at least 20 test rows; 28 answers in all):

When to use it

• You want a finer reading of feeling than positive or negative, for example gratitude, confusion or disappointment.
• Your texts are short, informal English comments or messages.
• You are happy to read the result as a spread of probabilities. Many comments carry more than one emotion.

When not to use it

• Your texts are long, formal or not in English. All training text is English Reddit comments.
• You need a clinical or safety judgement, such as risk of self-harm. The adapter only names emotions.
• You need every emotion in a text listed separately. The question asks for the one expressed most.

How to use it

The adapter runs with Jeff's server on the Jeff-Qwen3.5-0.8B v1.2 base. One server holds the base and any number of adapters; each request picks one by name with "model": "emotion".

git clone https://github.com/firelex/jeff && cd jeff
uv sync --no-default-groups --extra lora          # add --extra cuda on NVIDIA GPUs, --extra mac on Apple silicon
uv run --no-default-groups hf download mstrasser/Jeff-Qwen3.5-0.8B --revision v1.2 --local-dir Jeff-Qwen3.5-0.8B-v1.2
uv run --no-default-groups hf download mstrasser/Jeff-Qwen3.5-0.8B-emotion --local-dir adapters/emotion
JEFF_CHECKPOINT=Jeff-Qwen3.5-0.8B-v1.2 JEFF_ADAPTERS=adapters PORT=8765 \
  uv run --no-default-groups --extra lora jeff-serve
# add JEFF_BACKEND=mlx and --extra mac for Apple silicon

Every folder in adapters/ is served under its folder name. Add or replace an adapter while the server runs with POST /v1/adapters/reload. The server checks the base weights' checksum, so the adapter loads only on the exact v1.2 base.

from jeff import Client

emotion = Client("http://localhost:8765", model="jeff-latest").with_model("emotion")

Request format

State (the situation):

Questions:

• emotion (choice): Which emotion the comment expresses most strongly, or neutral if it expresses no particular emotion. Options: 28 options: 27 emotions and neutral, keyed by their GoEmotions names (admiration, amusement, anger and so on), each with a one-line description. The full list is GOEMOTIONS in descriptions.py in the source.

Rules:

• Use the option keys and descriptions from descriptions.py; the adapter was trained on them.
• Comments with several gold emotions were trained with the probability spread evenly over them, so a split answer is expected, not a fault.
• Use the instructions below word for word; the adapter was trained mostly on them.

Example

{
  "model": "emotion",
  "state": {
    "source": "Reddit comment ([NAME] and [RELIGION] replace removed names)",
    "comment": "Thanks so much for the tip, I had no idea the library lent out tools. Saved me a fortune."
  },
  "questions": {
    "emotion": {
      "type": "choice",
      "instructions": "Which emotion does the comment express most? Choose the emotion that is strongest in the comment, or neutral if it expresses no particular emotion.",
      "criteria": {
        "gratitude": "Gratitude: feeling thankful or appreciative.",
        "joy": "Joy: a feeling of pleasure and happiness.",
        "surprise": "Surprise: being astonished or startled by something unexpected.",
        "realization": "Realization: becoming aware of something.",
        "admiration": "Admiration: finding something impressive or worthy of respect.",
        "relief": "Relief: reassurance and relaxation after anxiety or distress ends.",
        "annoyance": "Annoyance: mild anger or irritation.",
        "confusion": "Confusion: lack of understanding or uncertainty.",
        "sadness": "Sadness: feeling sorrow or unhappiness.",
        "neutral": "Neutral: no particular emotion is expressed."
      }
    }
  }
}

curl -s localhost:8765/v1/systemone -H 'content-type: application/json' -d @request.json

The answer holds a probability for each option of each question, the chosen option and a confidence. (A recorded response to this example is not published yet.)

Training

The replay slice mixes in some of the base model's own training data so the adapter keeps the base's general skills. The scored checkpoint is the one at the end of the epoch.

Data and provenance

Sources:

Training data: built from the public sources above. A script to rebuild our rows from them will follow. The test set is included in this repo as test.jsonl.

Test set: test.jsonl, 5,408 rows: The official GoEmotions test split (simplified configuration); never trained on.

Calibration set: calibration.jsonl, 1,000 rows: the rows the threshold for the comparison with Qwen3.8-27B is chosen on; never trained on. It is byte for byte the set the adapter was scored on.

Quality review. An independent reviewer checked the training, calibration and test files for shortcuts, duplicates, leaks between splits and junk before training. The report is at jeffhub.ai/adapters/emotion/qa-report.

Limitations

• English only.
• Tied to the Jeff-Qwen3.5-0.8B v1.2 base. It will not work on any other base or version; the server refuses it if the base weights' checksum does not match. A v1.3 long-term-support base is coming, and the adapters will be retrained on it.
• Jeff chooses between the options you give it. It does not write text or reason in several steps.
• Calibration was fitted on this adapter's own calibration rows. On very different data, check it again.
• Accuracy is about 6 in 10 on GoEmotions. Use the probabilities, not only the top answer.
• Everything listed under When not to use it above.

Links

• Adapter page: jeffhub.ai/adapters/emotion
• Code and server: github.com/firelex/jeff
• Base model: mstrasser/Jeff-Qwen3.5-0.8B (revision v1.2)

Jeff is an independent project. It uses the same request format as Jev but is not affiliated with or endorsed by TypeSafe, the makers of Jev.