Qwen3.5-9B EAGLE3 Draft Model (Open-PerfectBlend, Decontaminated)
EAGLE3 draft model for Qwen3.5-9B, trained on a decontaminated instruction-mixture corpus. Use it with SGLang (or other EAGLE3-compatible runtimes) to accelerate autoregressive decoding via speculative sampling.
Model Overview
| Item | Value |
|---|---|
| Architecture | LlamaForCausalLMEagle3 (1-layer draft head) |
| Target model | Qwen3.5-9B |
| Draft vocab size | 50,000 |
| Aux hidden states | Layers 1, 15, 28 |
| Precision | bfloat16 |
Training Data
Training uses openeurollm/open-perfectblend-decontaminated, a decontaminated variant of mlabonne/open-perfectblend. Benchmark overlap (e.g. HumanEval, MATH, GPQA, LiveCodeBench) was removed before training to reduce evaluation contamination.
The dataset was converted to a single ShareGPT-style parquet file (role / content turns) for training.
Training Framework
Trained with mini-specforge (link), a lightweight EAGLE3 training pipeline built on PyTorch FSDP and Hugging Face Transformers.
| Hyperparameter | Value |
|---|---|
| Learning rate | 1e-4 |
| Batch size | 4 |
| Max sequence length | 2048 |
| Epochs | 1 |
| TTT length | 7 |
| Warmup ratio | 0.015 |
| Draft vocab mapping | Top-50k tokens from UltraChat (Qwen3.5 tokenizer) |
SGLang Evaluation
Benchmarked with SGLang EAGLE3 speculative decoding:
- Target model: Qwen3.5-9B
- Tensor parallel: 1 GPU (
--tp-size 1) - Attention backend: triton
- Spec config:
--spec-topk 1,--spec-num-steps∈ {3, 7, 15} - Benchmarks: MT-Bench, GSM8K, HumanEval (80 prompts each)
- Generation:
max_tokens=512, temperatures 0.0 and 1.0
Each cell reports accept length / speedup (Eagle3 throughput ÷ baseline throughput). The first column is the mean across the three datasets; the remaining columns are per-dataset results.
Temperature = 0.0
spec-num-steps |
Mean | MT-Bench | GSM8K | HumanEval |
|---|---|---|---|---|
| 3 | 2.97/2.25 | 2.45/1.92 | 3.26/2.43 | 3.18/2.39 |
| 7 | 3.97/2.59 | 2.80/2.00 | 4.62/2.90 | 4.48/2.88 |
| 15 | 4.34/2.19 | 2.84/1.59 | 5.08/2.41 | 5.09/2.56 |
Temperature = 1.0
spec-num-steps |
Mean | MT-Bench | GSM8K | HumanEval |
|---|---|---|---|---|
| 3 | 2.34/1.74 | 2.04/1.54 | 2.50/1.85 | 2.48/1.83 |
| 7 | 2.75/1.80 | 2.20/1.48 | 2.98/1.97 | 3.07/1.94 |
| 15 | 2.80/1.43 | 2.20/1.19 | 2.89/1.48 | 3.32/1.63 |
Usage (SGLang)
python -m sglang.launch_server \
--model-path Qwen/Qwen3.5-9B \
--speculative-algorithm EAGLE3 \
--speculative-draft-model-path <path-to-this-repo> \
--speculative-num-steps 7 \
--tp-size 1 \
Adjust --speculative-num-steps based on your latency/throughput trade-off (see table above).