Tencent's DSpine drafter for Qwen3.8-27B as a NInfer v3 artifact for NInfer-all (iamwavecut/ninfer-all). It is a speculative-decoding drafter, not a model of its own: each round it proposes up to 15 tokens, and the Qwen3.8-27B model it runs beside verifies them, so the answer stays the model's own. It loads beside any NInfer artifact of Qwen3.8-27B:
| Model | Artifact |
|---|---|
Qwen3.8-27B, the official groupwise-int artifact (measured below) |
neroued/Qwen3.8-27B-NInfer |
| Qwen3.8-27B GSQ-RCO IQ3_S | WaveCut/Qwen3.8-27B-GSQ-RCO-IQ3_S-NInfer-v3 |
| Fine-tunes of the same size and vocabulary (abliterated, HauhauCS, MXFP8-CRACK) | load it too; Tencent built the drafter for the base model, and its acceptance on a fine-tune is not measured |
NInfer refuses it beside a model of another hidden width, vocabulary or layer count. Only NInfer-all
reads DSpine drafters: build it from master, which has DSpine since pull request
#341. Stock NInfer builds cannot read them.
DSpine (Zhang et al., "Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding", arXiv 2609.36173) drafts a block of 16 positions in one pass. Five Qwen3 layers with an 8,192-token sliding window, causal inside the block, read the model's hidden states at layers 1, 16, 31, 46 and 61. After every layer a code bridge (1,024-dimensional codes, rank 512) writes into each position a message from the position before it, and a serial refinement over the head's 16 best candidates chains the drafts. The drafter has 2,025,912,064 parameters; it borrows the model's token embedding and output head.
Qwen3.8-27B-DSpine-b16-ninfer-v3.ninfer (2,434,027,008 bytes, 2.27 GiB) keeps the layer and
feature projections in 8-bit groups of 32 with FP16 scales and the code bridge and its tables in
BF16. SHA256SUMS lists its digest and that of its conversion record. From a NInfer-all checkout,
hf download tencent/Qwen3.8-27B-DSpine-b16 --revision 5f54174d301dbca65d9aa4448dc47e56732d9a12 \
--local-dir Qwen3.8-27B-DSpine-b16
python -m tools.convert --model Qwen3.8-27B-DSpine-b16 --recipe dspine_q8 --device cpu \
--out Qwen3.8-27B-DSpine-b16-ninfer-v3.ninfer
converts Tencent's checkpoint at that revision into this file. The dspine_bf16 recipe keeps every
weight in BF16 (4,056,043,008 bytes); on the RTX 5090 below, at seven drafts, it kept the same
tokens a round and decoded 9.1% and 7.8% slower on the code and prose answers (254.1 and 155.3 tok/s
against 279.5 and 168.5 in the same session), so this repository carries the Q8 file only.
Download it next to the model and pass it with --spec dspine --draft-model:
hf download WaveCut/Qwen3.8-27B-DSpine-b16-NInfer-v3 Qwen3.8-27B-DSpine-b16-ninfer-v3.ninfer \
--local-dir models
hf download neroued/Qwen3.8-27B-NInfer qwen3_8_27b.ninfer --local-dir models
ninfer-serve models/qwen3_8_27b.ninfer --model-id qwen3.8-27b --max-context 32768 \
--spec dspine --draft-model models/Qwen3.8-27B-DSpine-b16-ninfer-v3.ninfer --draft-tokens 7
A launch file takes spec = dspine, draft-model = PATH and draft-tokens = N. --draft-tokens
accepts 1 to 15: 15 paid on a long code answer, 7 on prose (measurements below). With eight
reasoning requests at once 7 drafts paid more than 15; with ninfer-serve --adaptive-mtp each round drafts
--draft-tokens or half of them. The drafter takes its 16 candidates from the model's full output
head; --lm-head-draft takes them from the artifact's proposal head, which kept fewer tokens a round
(5.14 against 5.43) and decoded slower. A sampled request verifies the drafter's sampled proposal by
rejection sampling, as DFlash2's.
Beside the model the drafter takes its 2.27 GiB of weights on the model's first GPU, a 242.5 MiB
table it derives from the model's output head at load (BF16, 512 × 248,320), and a ring of its own
keys and values for each request: 160 MiB at its 8,192-token window, which every --max-context
past 4,080 takes (40 MiB up to 2,032, 80 MiB up to 4,080). With the official artifact on an
RTX 5090 (16,384-token window, bf16 KV, one request) it left 2.7 GiB less free after startup than
no drafter (DFlash2 2.5 GiB, MTP 0.8 GiB), and the full 262,144-token window still fit with rk4v4
and rk8v4 KV.
One RTX 5090 at its 575 W default, the official groupwise-int artifact, the CLI with a
16,384-token window and bf16 KV, thinking off, one greedy request per configuration; decode tok/s,
with the tokens kept a round in parentheses. The Python answer continues a 1,024-token Python prompt,
the English one an English prose prompt. A NInfer-all build with the BF16 projection fix of
#346 (on master) is required for these
numbers at more than eight drafts.
| Answer | No drafts | MTP 3 | DFlash2 7 | DFlash2 15 | DSpine 7 | DSpine 11 | DSpine 15 |
|---|---|---|---|---|---|---|---|
| Python, 1,024 tokens | 90.7 | 251.8 (3.37) | 322.0 (4.55) | 405.1 (6.72) | 325.7 (4.92) | 421.5 (7.31) | 473.4 (8.38) |
| English, 1,024 tokens | 90.4 | 182.7 (2.45) | 187.8 (2.67) | 163.4 (2.73) | 172.9 (2.62) | 149.3 (2.60) | 156.0 (2.76) |
| code, 400 tokens | 90.3 | 232.3 (3.12) | 297.8 (4.20) | 292.7 (4.81) | 278.1 (4.20) | 251.1 (4.37) | 284.7 (4.99) |
| prose, 269 tokens | 90.3 | 185.2 (2.49) | 184.9 (2.61) | 150.4 (2.47) | 167.7 (2.54) | 151.8 (2.61) | 146.7 (2.56) |
On the long Python answer 15 drafts keep 8.38 tokens a round against DFlash2's 6.72 and decode 5.22
times as fast as plain decoding, 1.17 times as fast as DFlash2's best count. On the short code
answer and on prose DFlash2 with seven drafts stays 4% and 9% faster than the drafter's best count,
at nearly the same tokens a round: a DSpine round of seven drafts takes 15.2 ms of kernel time
against DFlash2's 14.2 ms, in its five layers with 17,408-wide MLPs and the full output head it
reads its candidates from. A sampled prose answer (temperature 0.7, seed 1) decoded 90.5 tok/s
without drafts, 210.2 with DFlash2 7 and 171.7 with DSpine 7. Four concurrent chat requests through
ninfer-serve with seven drafts each answered in 1.45 to 1.72 s (120 to 151 tokens), and a second
turn reused 148 of its 168 prompt tokens from the prefix cache.
Under the conditions of Tencent's model card (ninfer-serve on another RTX 5090 at its 600 W default, reasoning
effort xhigh, up to 2,048 tokens, prompts from GSM8K, MATH-500, HumanEval, MBPP and MT-Bench, no
copy rounds), 15 drafts kept the card's 7.20 tokens a round on GSM8K. Eight requests at once decoded
604.9 tok/s in all with 7 drafts, 480.1 with 15, 561.0 with --adaptive-mtp and 457.8 without
drafts; a round of 15 drafts verifies 128 columns at eight requests, 12.5 ms more than a round of 7.
DFlash2 decoded 629.1 with 7 drafts and 520.1 with 15
(measurements).
Every kept token is the model's own choice, but a verification's width changes floating-point results, so a greedy answer can leave the plain one, as MTP's and DFlash2's do. DSpine and DFlash2 rounds of the same width gave the same tokens on the Python, code and prose answers, and on the English one at seven drafts.
The drafter is Tencent's (Tencent/DSpine), released under the
Apache License 2.0, reproduced in LICENSE. This repository converts its weights into
NInfer's container, with the projections in 8 bits as described above.
@article{zhang2026dspine,
title = {Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding},
author = {Zhang, Haohui and Chen, Keyu and Sun, Haocheng and Gu, Weibo and Qiao, Ruizhi and Sun, Xing and Jiang, Bo},
journal = {arXiv preprint arXiv:2609.36173},
year = {2026}
}
tencent/Qwen3.8-27B-DSpine-b16