Woodcock drafter for Qwen3-4B

• Built by the team behind Atomic Chat, the app that runs open-weight models locally and offline.

---

A one-layer speculative-decoding drafter for Qwen/Qwen3-4B, for the Woodcock runtime on Apple Silicon (MLX). Training data, logs and benchmarks: AtomicChat/woodcock-training. Smaller sibling: AtomicChat/woodcock-drafter-qwen3-1.7b.

The drafter reads Qwen3-4B's own hidden states and guesses the next few tokens; Qwen3-4B checks all guesses in one forward pass and keeps the ones it agrees with. The output is Qwen3-4B's output: greedy decoding is token-for-token identical, and sampling reproduces its exact distribution. The drafter only changes the speed.

Speed

M4 Max (48 GB), MLX 0.32, bf16, batch 1, greedy, 256 new tokens per prompt, 10 prompts per domain (MT-Bench, HumanEval, GSM8K test, Aya). Woodcock with this drafter, a 16k-token draft head and automatic chain length (4.6 tokens on average), against mlx-lm stream_generate:

Held-out agreement with Qwen3-4B's next token, by draft step: 68.4%, 62.4%, 59.2%, 57.1%.

Usage

git clone https://github.com/AtomicBot-ai/woodcock.git && cd woodcock
pip install -e .
hf download AtomicChat/woodcock-drafter-qwen3-4b --local-dir drafters/qwen3-4b

# measure this machine's draft / verify latencies once (used by the automatic chain length)
python scripts/time_draft_step.py --target Qwen/Qwen3-4B --drafter drafters/qwen3-4b \
  --vocab drafters/qwen3-4b/vocab_counts.npz --sizes 16384 --out latency.json

woodcock generate --target Qwen/Qwen3-4B \
  --drafter drafters/qwen3-4b@drafters/qwen3-4b/vocab_counts.npz:16384 \
  --policy auto:latency.json \
  --prompt "Explain speculative decoding in two sentences."

@vocab_counts.npz:16384 restricts the drafter's LM head to the 16,384 tokens Qwen3-4B generates most often (FR-Spec, VocabTrim): a draft step becomes much cheaper and the verification still uses the full vocabulary, so the output does not change.

Files

Training

• Architecture: EAGLE-3-style drafter — fusion of three target layers (2, 18, 33) with per-layer RMSNorm, one decoder layer shaped like Qwen3-4B's, post-norm feedback (EAGLE-3.1), LM head over the full vocabulary.
• Objective: training-time test with 4 unrolled steps, soft-label cross-entropy to Qwen3-4B's distribution, step weights 0.8^k, loss on answer tokens only.
• Data: 80,000 prompts (chat 36k, code 18k, math 14k, multilingual 12k; sources in NOTICE), answered by Qwen3-4B itself (T = 0.7, top-p 0.8, top-k 20, ≤ 512 tokens, non-thinking chat template); 31.2M answer tokens, one pass.
• Optimisation: AdamW (β 0.9/0.95), lr 1e-4, 200 warm-up steps, cosine decay, clip 0.5, ~8k tokens per step, 6,400 steps, bf16 compute with fp32 master weights. One H100 (MLX CUDA backend), 1 h 55 min, peak 55 GB.

Limitations

• Built for Woodcock; it is not a standalone language model and does not load in transformers.
• Thinking mode (<think> reasoning) was not in the training data; speedups there are not measured.
• In bf16, a batched verification pass can resolve a near-tie between two tokens differently from a one-token pass, so long greedy outputs can diverge from plain bf16 decoding at such a tie (identical in fp32).

License

Apache-2.0. The weights are derived from Qwen3-4B (Apache-2.0); see LICENSE and NOTICE. The Woodcock code is MIT.