Woof Pup 0.6B — email triage (Core ML, Apple Neural Engine)
The Neural Engine build of alexwengg/woof-pup-0.6b-mlx-8bit, a 0.6B email-triage model distilled from ConwayResearch/Underdog-Woof-4B-1.1. Email in, one JSON object out (category, intent, action items, dates, amount, urgency). Converted with ANEMLL.
What runs where
Of the operations Core ML actually executes, 99.4% run on the Neural Engine (7,190 ops). The CPU handles only the embedding lookup, a few KV-cache position and mask ops, and the final next-word choice, which has no Neural Engine op.
Speed (Apple M5 Pro)
| Neural Engine (this) | GPU, MLX 8-bit | |
|---|---|---|
| Generation | ~80 tok/s | ~240 tok/s |
| Prompt processing | ~550–650 tok/s | — |
| First token (~290-token email) | ~560 ms | ~35 ms |
| GPU used | none | yes |
The Neural Engine is slower per token here because decode is memory-bandwidth bound. It runs at a few watts with the GPU idle. For raw speed use the MLX build; for low power or a free GPU, use this one.
Quality
Against Woof 4B's labels on 291 held-out emails (2 of 293 exceed the 1,024-token context):
| Field | Agreement with Woof 4B |
|---|---|
| Valid JSON | 100% |
| Category | 71.5% |
| Urgency | 87.3% |
| Amount | 93.8% |
That is within 1.5 points of the MLX build, so the 6-bit conversion costs essentially nothing.
Files
| File | What |
|---|---|
qwen_embeddings.mlmodelc |
token embeddings |
qwen_FFN_PF_lut6_chunk_01of01.mlmodelc |
all 28 transformer layers, prefill + decode functions, 6-bit LUT weights |
qwen_lm_head_lut6.mlmodelc |
output layer, 6-bit LUT |
*.mlpackage |
the same three models uncompiled, for Xcode |
meta.yaml |
ANEMLL config: context 1,024, prefill batch 64, LM head split 16 |
| tokenizer files | Qwen3 tokenizer |
Usage
With ANEMLL's Python runner:
git clone https://github.com/Anemll/Anemll && cd Anemll && ./create_uv_env.sh && ./install_dependencies.sh
hf download alexwengg/woof-pup-0.6b-coreml --local-dir ../woof-pup-coreml
python tests/chat.py --meta ../woof-pup-coreml/meta.yaml --no-template --prompt "$PROMPT" --max-tokens 250
$PROMPT must be the full training-format prompt (ChatML with the system prompt below, the email prefixed with EMAIL:\n, and an empty thinking block):
<|im_start|>system
{system prompt}<|im_end|>
<|im_start|>user
EMAIL:
{email}<|im_end|>
<|im_start|>assistant
<think>
</think>
From Swift, ANEMLL's AnemllCore package loads meta.yaml directly (YAMLConfig, ModelLoader, InferenceManager).
The SwiftUI demo in the MLX repo shows the calls.
scripts/ane_eval.py scores the model on held-out emails; scripts/ane_placement.py prints the per-op device placement
from Core ML's compute plan, with no Xcode needed.
You are an email triage engine. Read the email and return ONLY a JSON object, no prose, no markdown fences, with exactly these keys:
"category": one of "request","fyi","scheduling","approval","negotiation","personal","announcement","other"
"intent": one sentence, max 20 words, what the sender wants
"action_items": array of short strings, things the recipient must do (empty array if none)
"dates": array of date or deadline mentions as written in the email (empty array if none)
"amount": {"value": number, "currency": "USD"} if a money amount is mentioned, otherwise null
"urgency": one of "low","medium","high"
Conversion notes
ANEMLL with --lut2 6 --context 1024 --chunk 1. 4-bit LUT weights broke this small model's JSON output on the GPU side, so 6-bit is used.
Compiled with coremltools.models.utils.compile_model (no Xcode). Requires macOS 15 / iOS 18 or later.
Core ML on the GPU is not supported for this build: the 64-token batched prefill fails on the Metal backend.
License
Apache-2.0, as are the base model (Qwen3-0.6B) and the teacher (Woof 4B). Training emails come from the public AESLC dataset built on the Enron email corpus.