DSpine for Qwen3.8-27B
DSpine generates draft tokens in parallel. Each layer passes predicted predecessor features to adjacent positions, carrying causal dependencies through network depth.
This checkpoint is a five-layer draft model for Qwen3.8-27B, with block size 16 and an 8,192-token sliding window.
Evaluation
- Runtime: SGLang 0.5.22, BF16, FlashAttention 3, concurrency 8.
- Draft blocks: DSpine 16; official DFlash2 8.
- DSpine inference: dynamic verification referencing DSpark, with ReplaySSM enabled.
- General benchmarks:
xhighreasoning, greedy and temperature sampling, at most 4,096 new tokens. - Agent benchmarks: We evaluate WorkBuddy Bench (Code/Web/Office) using CodeBuddy sandbox trajectories:
low/xhighreasoning, temperature sampling, 20 conversations per subset, all tool-call prefixes replayed in order, prefix caching, and up to 32,768 new tokens per request. - Temperature sampling:
temperature=1.0,top_p=0.95,top_k=20.
Results
General Benchmark Acceptance Length
| Task | DFlash2(T=0) | DSpine(T=0) | DFlash2(T=1) | DSpine(T=1) |
|---|---|---|---|---|
| GSM8K | 5.69 | 7.20 | 5.45 | 6.31 |
| MATH-500 | 5.49 | 6.49 | 5.27 | 5.66 |
| HumanEval | 4.87 | 5.71 | 4.34 | 4.71 |
| MBPP | 5.04 | 6.14 | 4.79 | 5.41 |
| MT-Bench | 4.11 | 4.48 | 3.89 | 4.03 |
Agent Acceptance Length
| Benchmark | low | xhigh | ||
|---|---|---|---|---|
| DFlash2 | DSpine | DFlash2 | DSpine | |
| WorkBuddy Code | 4.19 | 5.08 | 4.06 | 4.92 |
| WorkBuddy Web | 3.95 | 4.91 | 3.95 | 5.01 |
| WorkBuddy Office | 4.24 | 5.63 | 4.20 | 5.49 |
| SWE-bench Pro | 4.19 | 4.99 | 4.00 | 4.77 |
| τ²-bench | 4.70 | 5.53 | 4.29 | 4.95 |
Agent End-to-End Throughput
Throughput is total output tokens divided by end-to-end concurrent replay wall time (tokens/s). Parentheses show speedup over autoregressive decoding (AR).
| Benchmark | low | xhigh | ||||
|---|---|---|---|---|---|---|
| AR | DFlash2 | DSpine | AR | DFlash2 | DSpine | |
| WorkBuddy Code | 253.35 | 590.27 (2.33×) | 686.23 (2.71×) | 193.00 | 550.40 (2.85×) | 613.55 (3.18×) |
| WorkBuddy Web | 247.40 | 639.74 (2.59×) | 773.19 (3.13×) | 252.26 | 734.04 (2.91×) | 840.14 (3.33×) |
| WorkBuddy Office | 210.03 | 664.77 (3.17×) | 805.53 (3.84×) | 215.05 | 662.34 (3.08×) | 763.83 (3.55×) |
| SWE-bench Pro | 241.86 | 657.71 (2.72×) | 713.41 (2.95×) | 248.17 | 711.60 (2.87×) | 791.30 (3.19×) |
| τ²-bench | 301.39 | 891.37 (2.96×) | 964.30 (3.20×) | 340.39 | 871.98 (2.56×) | 978.39 (2.87×) |
Installation
git clone https://github.com/Tencent/DSpine.git
cd DSpine
pip install -e .
Usage
sglang serve \
--model-path Qwen/Qwen3.8-27B \
--speculative-algorithm DSPINE \
--speculative-draft-model-path /path/to/Qwen3.8-27B-DSpine-b16 \
--speculative-num-draft-tokens 16 \
--dtype bfloat16 \
--attention-backend fa3 \
--speculative-draft-attention-backend fa3 \
--linear-attn-backend triton \
--enable-linear-replayssm-spec \
--mamba-ssm-dtype float32 \
--mamba-radix-cache-strategy extra_buffer \
--max-mamba-cache-size 64 \
--mem-fraction-static 0.80 \
--context-length 131072 \
--max-running-requests 8 \
--cuda-graph-max-bs-decode 8 \
--min-free-slots-delay 1 \
--sampling-backend flashinfer \
--reasoning-parser qwen3 \
--tool-call-parser qwen3_coder \
--trust-remote-code
Acknowledgements
We thank DFlash, SpecForge, SGLang and FlashInfer. Dynamic verification references DSpark.
Citation
@article{zhang2026dspine,
title = {Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding},
author = {Zhang, Haohui and Chen, Keyu and Sun, Haocheng and Gu, Weibo and Qiao, Ruizhi and Sun, Xing and Jiang, Bo},
journal = {arXiv preprint arXiv:2609.36173},
year = {2026}
}