Qwen3.5-4B EAGLE3 Draft Model (Open-PerfectBlend, Decontaminated)
EAGLE3 draft model for Qwen3.5-4B, trained on a decontaminated instruction-mixture corpus. Use it with SGLang (or other EAGLE3-compatible runtimes) to accelerate autoregressive decoding via speculative sampling.
Model Overview
| Item | Value |
|---|---|
| Architecture | LlamaForCausalLMEagle3 (1-layer draft head) |
| Target model | Qwen3.5-4B |
| Draft vocab size | 50,000 |
| Aux hidden states | Layers 1, 15, 28 |
| Precision | bfloat16 |
Training Data
Training uses openeurollm/open-perfectblend-decontaminated, a decontaminated variant of mlabonne/open-perfectblend. Benchmark overlap (e.g. HumanEval, MATH, GPQA, LiveCodeBench) was removed before training to reduce evaluation contamination.
The dataset was converted to a single ShareGPT-style parquet file (role / content turns) for training.
Training Framework
Trained with mini-specforge (link), a lightweight EAGLE3 training pipeline built on PyTorch FSDP and Hugging Face Transformers.
| Hyperparameter | Value |
|---|---|
| Learning rate | 1e-4 |
| Batch size | 6 |
| Max sequence length | 2048 |
| Epochs | 2 |
| TTT length | 7 |
| Warmup ratio | 0.015 |
| Draft vocab mapping | Top-50k tokens from UltraChat (Qwen3.5-4B tokenizer) |
SGLang Evaluation
Benchmarked with SGLang EAGLE3 speculative decoding:
- Target model: Qwen3.5-4B
- Tensor parallel: 1 GPU (
--tp-size 1) - Attention backend: triton
- Spec config:
--spec-topk 1,--spec-num-steps∈ {3, 7, 15} - Benchmarks: MT-Bench, GSM8K, HumanEval (80 prompts each)
- Generation:
max_tokens=512, temperatures 0.0 and 1.0
Each cell reports accept length / speedup (Eagle3 throughput ÷ baseline throughput). The first column is the mean across the three datasets; the remaining columns are per-dataset results.
Temperature = 0.0
spec-num-steps |
Mean | MT-Bench | GSM8K | HumanEval |
|---|---|---|---|---|
| 3 | 2.76/1.91 | 2.32/1.66 | 3.08/2.10 | 2.88/1.98 |
| 7 | 3.50/2.10 | 2.61/1.68 | 4.19/2.42 | 3.71/2.21 |
| 15 | 3.66/1.70 | 2.66/1.32 | 4.41/1.95 | 3.92/1.82 |
Temperature = 1.0
spec-num-steps |
Mean | MT-Bench | GSM8K | HumanEval |
|---|---|---|---|---|
| 3 | 2.29/1.53 | 1.99/1.38 | 2.43/1.62 | 2.45/1.60 |
| 7 | 2.61/1.51 | 2.11/1.26 | 2.91/1.69 | 2.82/1.56 |
| 15 | 2.72/1.23 | 2.16/0.98 | 2.82/1.26 | 3.19/1.44 |
Usage (SGLang)
python -m sglang.launch_server \
--model-path Qwen/Qwen3.5-4B \
--speculative-algorithm EAGLE3 \
--speculative-draft-model-path <path-to-this-repo> \
--speculative-num-steps 7 \
--tp-size 1 \
Adjust --speculative-num-steps based on your latency/throughput trade-off (see table above).