Woof Pup 0.6B — email triage (MLX, 8-bit)

A 0.6B email-triage model distilled from ConwayResearch/Underdog-Woof-4B-1.1. Give it an email, get back one JSON object:

{"category": "request", "intent": "Request a call to discuss a report modification",
 "action_items": ["Call Cara Semperger or the sender"], "dates": [], "amount": null, "urgency": "medium"}

Categories: request, fyi, scheduling, approval, negotiation, personal, announcement, other. Urgency: low, medium, high.

Neural Engine (Core ML) build: alexwengg/woof-pup-0.6b-coreml.

Speed (Apple M5 Pro, 24 GB)

Quality

Scored against Woof 4B's labels on 293 held-out emails never seen in training (8-bit weights):

Category labels are fuzzy: asked the same email twice with a reworded prompt, Woof 4B itself agrees on category only 78–84% of the time. On the emails where its two answers agree, this model matches it 81.5% on category and 90.3% on urgency.

Usage

pip install mlx-lm

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tok = load("alexwengg/woof-pup-0.6b-mlx-8bit")
SYSTEM = open("demo/demo_system_prompt.txt").read()   # exact prompt used in training (also below)
email = "Can we move Thursday's review to Friday 3pm? Need sign-off on the $12,000 budget before then."
prompt = tok.apply_chat_template(
    [{"role": "system", "content": SYSTEM}, {"role": "user", "content": "EMAIL:\n" + email}],
    add_generation_prompt=True, tokenize=False, enable_thinking=False)
print(generate(model, tok, prompt=prompt, max_tokens=300, sampler=make_sampler(temp=0.0)))

Use the system prompt verbatim, prefix the email with EMAIL:\n, disable thinking, and decode greedily. That is the format it was trained on.

System prompt

You are an email triage engine. Read the email and return ONLY a JSON object, no prose, no markdown fences, with exactly these keys:
"category": one of "request","fyi","scheduling","approval","negotiation","personal","announcement","other"
"intent": one sentence, max 20 words, what the sender wants
"action_items": array of short strings, things the recipient must do (empty array if none)
"dates": array of date or deadline mentions as written in the email (empty array if none)
"amount": {"value": number, "currency": "USD"} if a money amount is mentioned, otherwise null
"urgency": one of "low","medium","high"

Demo app (macOS, SwiftUI)

demo/ is a native Mac app that triages 200 held-out emails one after another: live queue, streaming result card, timing per email.

cd demo && ./run_demo.sh     # sets up mlx-lm + ANEMLL, serves this model on :8082, builds and opens the app
./monitor.sh                 # optional: macmon + live server log in one terminal (brew install tmux macmon)

Needs an Apple silicon Mac on macOS 15+, the Xcode Command Line Tools and Python 3.10+.

How it was made

1. Teacher labels. Woof 4B labelled 14.6k emails from the AESLC (Enron) corpus in the JSON format above; ~1% malformed outputs were dropped, leaving 14.5k labels.
2. Student. Qwen3-0.6B, full fine-tune of all transformer layers on 13.7k labelled emails (4,500 steps, cosine learning rate peaking at 5e-5), with the 293 evaluation emails held out.
3. Soft-label distillation. 3,000 more steps adding a KL loss toward Woof 4B's probability distribution over the 8 categories and 3 urgency levels (weight 2). This lifted category agreement from 70.6% to 73.0% and urgency from 83.3% to 85.3%.
4. Quantization. 8-bit MLX. 8-bit matched full precision; 4-bit broke the model (repetition loops, invalid JSON).

Trained on a single 24 GB M5 Pro MacBook with mlx-lm.

Limits

• English business email, mostly 2000-era Enron mail. Expect weaker results on very different writing.
• It reproduces Woof 4B's judgement, including its inconsistencies on borderline categories.
• Intent and action items are short summaries, not extraction-grade quotes.

License

Apache-2.0, as are the base model (Qwen3-0.6B) and the teacher (Woof 4B). Training emails come from the public AESLC dataset built on the Enron email corpus.