Qwen3-1.7B-DSpark
A DSpark draft model for speculative decoding with Qwen/Qwen3-1.7B as the verifier, trained with speculators. The drafter proposes 8 tokens at a time and the verifier checks them in one forward pass, so output is identical to running the verifier alone — a lossless speedup. Mean acceptance length is 3.75 tokens committed per verification step, up to 5.01 on math_reasoning. That gives a throughput speedup over the verifier alone of 3.30× on math_reasoning and 2.67× on HumanEval, and 2.35× averaged across nine task types.
Training code: rasyosef/train-dspark-draft-models.
Trained on 100,000 samples.
Usage
vLLM
vLLM loads the verifier automatically from the config — don't pass it separately.
vllm serve rasyosef/Qwen3-1.7B-DSpark --port 8000 --gpu-memory-utilization 0.8
Then query the OpenAI-compatible endpoint at http://localhost:8000/v1.
llama.cpp
llama-server -hf unsloth/Qwen3-1.7B-GGUF:Q8_0 \
-hfd rasyosef/Qwen3-1.7B-DSpark-GGUF:Q8_0 \
--spec-type draft-dspark --spec-draft-n-max 8 --spec-draft-n-min 0 \
-fa on -ngl 99
The block size is read from the sidecar metadata (n-max is clamped to it). Speculative decoding is exact: the target verifies every proposed token, so greedy output equals the target alone; per-response timings report draft_n / draft_n_accepted.
Details
4 Qwen3 layers (hidden size 2048, intermediate size 6144, 16 attention heads over 8 KV heads, head dim 128). The first three layers use sliding-window attention with a 2048-token window and the last layer uses full attention. Block size 8, aux hidden-state layers 2/10/18/26.
Trained for 3 epochs at lr 3e-4 (cosine schedule with 4% warmup) on 100,000 Open PerfectBlend prompts regenerated by the verifier itself, split 96/4 into train and validation, with a {"ce": 0.1, "tv": 0.9} loss. Prompts prepared at 2048 tokens; training sequence length 8192, up to 1024 anchors per sample. Verifier hidden states were pulled on demand from a running vLLM server during training and deleted after use rather than staged to disk up front.
Evaluation
evaluate.py throughput across the nine RedHatAI/speculator_benchmarks subsets. acceptance_length is mean tokens committed per verification step, including the bonus token — floor 1.0, ceiling 9.0 at block size 8.
| subset | acceptance_length | pos_0 | pos_1 | pos_2 | pos_3 | pos_4 | pos_5 | pos_6 | pos_7 |
|---|---|---|---|---|---|---|---|---|---|
| math_reasoning | 5.010 | 85.8% | 72.5% | 61.4% | 51.4% | 42.9% | 35.6% | 28.7% | 22.8% |
| HumanEval | 4.201 | 82.2% | 65.7% | 51.5% | 39.7% | 30.2% | 22.5% | 16.5% | 11.7% |
| translation | 3.468 | 76.3% | 57.2% | 41.1% | 27.8% | 18.8% | 12.5% | 7.9% | 5.0% |
| rag | 3.439 | 74.8% | 54.8% | 39.7% | 28.1% | 19.5% | 13.0% | 8.6% | 5.4% |
| question | 3.429 | 73.6% | 52.8% | 38.1% | 27.4% | 19.7% | 14.2% | 10.1% | 7.0% |
| writing | 3.426 | 73.8% | 52.8% | 37.8% | 27.3% | 19.7% | 14.1% | 10.0% | 7.1% |
| qa | 3.377 | 72.3% | 51.9% | 36.8% | 27.0% | 19.3% | 13.6% | 9.9% | 7.0% |
| tool_call | 3.247 | 72.8% | 51.4% | 35.7% | 24.6% | 16.8% | 11.2% | 7.5% | 4.8% |
| summarization | 2.641 | 66.5% | 41.4% | 24.9% | 14.2% | 8.3% | 4.8% | 2.5% | 1.4% |
Weighted across all subsets: 3.749 over 357,424 verification steps.
Acceptance is highest where the verifier's next token is most predictable — math and code. math_reasoning leads HumanEval by about 0.8 tokens and holds a margin at every position in the block; math_reasoning's pos_7 (22.8%) is nearly double HumanEval's (11.7%), and its pos_4 (42.9%) is above summarization's pos_1 (41.4%). translation, rag, question, and writing cluster between 3.43 and 3.47, with qa just behind at 3.38. Unlike many drafters, tool_call does not stand apart here: at 3.25 it sits at the bottom of the prose-like group rather than above it.
Throughput (tokens/s)
Mean output throughput (tokens/s) across the same nine subsets, measured on a single A100 at concurrency 1. Baseline is Qwen3-1.7B with no speculative decoding, measured on every subset; without a drafter, throughput barely depends on the task (218.7–223.3 tokens/s). Speedup is the mean throughput relative to that subset's baseline.
| subset | baseline (no drafter) | DSpark (this model) | speedup |
|---|---|---|---|
| math_reasoning | 220.1 | 725.8 | 3.30× |
| HumanEval | 219.7 | 586.5 | 2.67× |
| translation | 223.3 | 555.9 | 2.49× |
| rag | 218.7 | 506.0 | 2.31× |
| question | 221.3 | 476.0 | 2.15× |
| writing | 220.9 | 483.8 | 2.19× |
| qa | 222.5 | 477.3 | 2.15× |
| tool_call | 219.9 | 471.7 | 2.15× |
| summarization | 219.4 | 384.5 | 1.75× |
| average speedup | 1.00× | — | 2.35× |
DSpark averages a 2.35× speedup (unweighted mean of per-subset speedups). The biggest gains are on math_reasoning at 3.30× (220.1 → 725.8 tokens/s) and HumanEval at 2.67×. translation is next at 2.49× and rag at 2.31×; writing, question, qa, and tool_call sit in a tight band between 2.15× and 2.19×. summarization is lowest at 1.75×.
Means and medians agree to within about 5% on every subset. The gap is widest on writing, question, and tool_call, where the mean runs 4–5% above the median, so a typical request on those subsets sees closer to 2.05×.
Limitations
Works only with Qwen3-1.7B and is not usable as a standalone model. Acceptance falls off steeply past the first few positions on prose-like traffic, and on summarization in particular a block size of 8 is mostly wasted — the gains concentrate in math and code. Tool-call traffic sees only prose-level acceptance with this drafter. Throughput was measured at concurrency 1 on one A100; real-world speedup depends on your traffic mix, hardware, and load, and because verification is lossless, the verifier's own behavior and biases carry through unchanged.
License
Apache-2.0, matching the verifier. The speculators training code is also Apache-2.0.