Qwen3.5-2B-DFlash2
A DFlash2 draft model for speculative decoding with Qwen/Qwen3.5-2B as the verifier, trained with speculators. With a block size of 8, the drafter proposes 7 tokens at a time and the verifier checks them in one forward pass, so output is identical to running the verifier alone — a lossless speedup. At temperature 0, mean acceptance length is 3.49 tokens committed per verification step, up to 5.07 on math_reasoning.
Training code: rasyosef/train-dspark-draft-models.
Trained on 100,000 samples.
Usage
vLLM loads the verifier automatically from the config — don't pass it separately.
vllm serve rasyosef/Qwen3.5-2B-DFlash2 --port 8000 \
--gpu-memory-utilization 0.8 \
--override-generation-config '{"temperature": 0}'
Then query the OpenAI-compatible endpoint at http://localhost:8000/v1.
Details
4 Qwen3 layers (hidden size 2048, intermediate size 6144, 16 attention heads over 4 KV heads, head dim 128), with the first three layers using sliding-window attention (2048-token window) and the last using full attention. Block size 8 (7 draft tokens per step), aux hidden-state layers 1/5/9/13/17/21, bfloat16.
Trained for 4 epochs at lr 4e-4 (AdamW, cosine schedule with 4% warmup) on 100,000 filtered Open PerfectBlend prompts with responses regenerated by the verifier itself, split 96/4 into train and validation, with a {"ce": 0.1, "tv": 0.9} loss. Samples prepared at 1024 tokens; training sequence length 4096, up to 512 anchors per sample. Verifier hidden states were pulled on demand from a running vLLM server during training and deleted after use rather than staged to disk up front.
Evaluation
evaluate.py throughput across the nine RedHatAI/speculator_benchmarks subsets, run at temperature 0 (greedy decoding). acceptance_length is mean tokens committed per verification step, including the bonus token — floor 1.0, ceiling 8.0 at block size 8 (7 draft tokens).
| subset | acceptance_length | pos_0 | pos_1 | pos_2 | pos_3 | pos_4 | pos_5 | pos_6 |
|---|---|---|---|---|---|---|---|---|
| math_reasoning | 5.069 | 88.3% | 76.0% | 64.8% | 55.5% | 47.3% | 40.4% | 34.6% |
| HumanEval | 4.361 | 82.3% | 66.5% | 54.4% | 44.3% | 36.3% | 29.5% | 22.8% |
| rag | 3.778 | 79.4% | 59.7% | 45.0% | 34.9% | 27.4% | 19.4% | 12.0% |
| qa | 3.217 | 65.5% | 43.4% | 31.1% | 24.7% | 21.6% | 18.7% | 16.7% |
| tool_call | 3.077 | 67.5% | 45.3% | 30.9% | 23.2% | 17.8% | 13.4% | 9.6% |
| writing | 2.977 | 66.2% | 42.2% | 28.1% | 21.0% | 16.6% | 13.2% | 10.4% |
| question | 2.811 | 64.9% | 40.8% | 26.0% | 18.3% | 13.6% | 10.2% | 7.3% |
| summarization | 2.432 | 64.7% | 36.7% | 19.9% | 10.9% | 6.1% | 3.2% | 1.7% |
| translation | 2.265 | 65.8% | 35.1% | 15.8% | 6.2% | 2.5% | 0.8% | 0.3% |
Weighted across all subsets (temperature 0): 3.488 over 127,690 verification steps.
Acceptance is highest where the verifier's next token is most predictable. math_reasoning is the only subset above 5.0 and still accepts more than a third of drafts at pos_6 (34.6%); HumanEval follows at 4.36 and stays above 20% through pos_6. rag (3.78) is third: it starts close to HumanEval at pos_0 (79.4%) but decays faster, to 12.0% at pos_6. The next four — qa, tool_call, writing and question — sit in a 2.81–3.22 band, with pos_0 acceptance of about 65–67%. qa (3.22) leads that band on an unusually flat tail: it still accepts 16.7% at pos_6, the third-highest of any subset and ahead of rag. tool_call (3.08) is only slightly ahead of open-ended writing (2.98), so the structure of a tool call is apparently not buying much predictability here. summarization (2.43) and translation (2.26) are lowest; both start level with the middle band at pos_0 (64.7% and 65.8%) but collapse quickly, translation to 15.8% at pos_2 and under 1% from pos_5 onward. translation is also the smallest subset (917 steps), so treat its number as noisy.
First-position acceptance spans 65–88% across all nine subsets, and seven of the nine remain above 10% at pos_4 (all but summarization and translation). Overall, 35.5% of drafted tokens are accepted (317,702 of 893,830).
Limitations
Works only with Qwen3.5-2B and is not usable as a standalone model. The verifier is vision-language, but the drafter was trained on text-only prompts and was evaluated on text-only benchmarks, so acceptance on image inputs is untested. Acceptance still falls with position: about 64% of drafted tokens are rejected overall, and on summarization and translation fewer than 7% of drafts survive to pos_4, so the tail of each 7-token draft is largely wasted on that traffic and gains there are likely smaller. The acceptance figures above are for temperature 0. Throughput has not been benchmarked for this checkpoint; real-world speedup depends on your hardware and traffic mix, and because verification is lossless, the verifier's own behavior and biases carry through unchanged.
License
Apache 2.0, inherited from the verifier. The speculators training code is Apache-2.0.