Tess-4-27B · EAGLE-3 Draft
An EAGLE-3 speculative-decoding draft head for migtissera/Tess-4-27B — trained on-policy against Tess-4's own outputs, so it predicts this model, not its base.
Measured: 1.76× faster decoding on average — up to 2.4× on reasoning. Lossless: the target model verifies every token, so outputs are identical to non-speculative decoding.
Benchmarks
Single H100-80GB, greedy decoding, 10 held-out prompts × 1024 tokens each:
| Workload | Baseline | With EAGLE-3 | Speedup |
|---|---|---|---|
| Math / logic reasoning | 51.2 tok/s | 103–123 tok/s | 2.0–2.4× |
| Coding | 51.2 tok/s | 81–91 tok/s | 1.6–1.8× |
| Agentic planning | 51.2 tok/s | 83 tok/s | 1.6× |
| General chat | 51.2 tok/s | 80–87 tok/s | 1.6–1.7× |
| Overall | 51.2 tok/s | 90.2 tok/s | 1.76× |
Server-reported accept length: 1.5–2.2 tokens per verification step (conservative settings: topk=1, 4 draft tokens — tuning headroom remains).
Why this exists
Tess-4-27B inherits an MTP head from its base model, but fine-tuning shifts the output distribution, driving that head's acceptance to ~0% — it can't accelerate the fine-tune. This EAGLE-3 draft was trained against Tess-4 itself: ~4,700 on-policy samples (Tess-4's own reasoning + coding + agentic generations, including real multi-turn tool-call traces), 2 epochs, with the draft consuming fused hidden states from three depths of the frozen target.
Usage
SGLang (tested ✅)
python -m sglang.launch_server \
--model-path migtissera/Tess-4-27B \
--speculative-algorithm EAGLE3 \
--speculative-draft-model-path migtissera/Tess-4-27B-EAGLE3 \
--speculative-num-steps 3 \
--speculative-eagle-topk 1 \
--speculative-num-draft-tokens 4 \
--trust-remote-code
vLLM
vllm serve migtissera/Tess-4-27B \
--speculative-config '{"method": "eagle3", "model": "migtissera/Tess-4-27B-EAGLE3", "num_speculative_tokens": 4}' \
--trust-remote-code
(Benchmarked with SGLang; vLLM flags may vary by version.)
Details
- Architecture: 1-layer Llama-style EAGLE-3 head (hidden 5120, matching the target), draft vocab 32K mapped onto the target's 248K vocab. 1.22 GB. Input embeddings are not included — the serving engine shares them from the target model.
- Training: SpecForge (SGLang team), online mode, frozen target, single H100.
- Data: on-policy — the target generated every training token. Mix: agentic/coding prompts from the Tess-4 program, OpenCoder, GSM8K, UltraChat, plus real multi-turn tool-call traces.
- Lossless: speculative decoding never changes outputs; the target verifies every proposed token. Speedup depends on workload (structured/reasoning content accepts more).
License
Apache 2.0, matching Tess-4-27B.
Part of the Tess series by Migel Tissera.