Qwen3.8-27B-DSpine-b16 GGUF

GGUF conversions of tencent/Qwen3.8-27B-DSpine-b16, the DSpine speculative-decoding draft model for Qwen3.8-27B, for use with llama.cpp.

On a single RTX 3090 with a Q4_K_M Qwen3.8-27B target, it runs at ~125 tok/s on code and math and ~90 tok/s on a mixed set of chat tasks. That compares with ~52 tok/s for DFlash2 and ~28 tok/s without speculative decoding (3.2x overall, up to 4.5x on code). See Benchmarks for speed at increasing context depth and the maximum context on 24 GB.

The speedup depends on your GPU. On AMD, Intel and bandwidth-limited systems the best settings differ, so check Other GPUs and tuning before you start.

[!IMPORTANT] Upstream llama.cpp does not support DSpine yet. These files need the dspine branch of this fork: https://github.com/Ajay9o9/llama.cpp/tree/dspine

The branch is current llama.cpp master plus DSpine support. Stock llama.cpp refuses to load these files rather than running them incorrectly.

Files

These files contain the draft model only. You also need a Qwen3.8-27B target GGUF, such as Q4_K_M.

What DSpine is

DSpine extends DFlash. It is a five-layer drafter that reads hidden states from five layers of the target model and drafts a block of 16 tokens in parallel. It adds three things:

• Causal block attention. Each draft position attends only to earlier positions in its block.
• A code bridge after every layer. Each position receives a gated message built from its predecessor's predicted token code. This passes causal dependencies through the depth of the network.
• Serial last-write refinement. For the top 16 candidates at each position, the drafter redoes the last bridge write using each candidate predecessor's exact code. This produces a 16x16 transition table per position. The speculative decoder walks the table from the last accepted token to pick the draft chain, either greedily or by sampling.

See the paper and the original model card for details.

Build

git clone -b dspine https://github.com/Ajay9o9/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j

Run

./build/bin/llama-server \
  -m  Qwen3.8-27B-Q4_K_M.gguf \
  -md Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 15 \
  -ngl 999 --spec-draft-ngl 999 \
  -fa on --jinja -c 32768 -np 1 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --spec-draft-type-k q8_0 --spec-draft-type-v q8_0

• Set --spec-draft-n-max yourself. The default is 3, which uses only a fraction of the 16-token block. 15 is the best value on NVIDIA CUDA, but on some GPUs a smaller value is faster. See Other GPUs and tuning.
• Set -c yourself instead of relying on auto-fit. Auto-fit does not leave enough room for the drafter, and with a vision projector (--mmproj) loaded the first image request can crash. If the log shows failed to allocate ... using host copies, lower -c.
• --spec-draft-sampling probabilistic is optional. It samples the draft chain from the transition table, and helps slightly at temperature > 0.
• Context on a 24 GB card: with a Q4_K_M target and q8_0 KV cache, DSpine at 15 drafted tokens fits up to -c 65536. For more context, lower --spec-draft-n-max (at 7 it fits up to -c 98304), use a smaller target quant, or use a larger GPU. See Maximum context.
• On 24 GB cards with -np 2, use --spec-draft-n-max 7. Two slots with 15-token drafts do not fit.

Other GPUs and tuning

How fast DSpine is depends on how quickly your GPU can verify a block of 16 tokens at once. On an RTX 3090 that step is cheap, so long drafts pay off. On GPUs where small batches are slow, a 15-token draft can cost more than it saves.

Results from the community (thank you!):

What to do:

• Sweep --spec-draft-n-max on your own prompts: try 2, 3, 4, 7 and 15. Starting points:
  • NVIDIA (CUDA) and AMD on ROCm / HIP: 15.
  • AMD on Vulkan: 2 or 3.
  • Intel Arc: 2 to 4. Users report it is slower than DFlash2 at 15, but there are no measured numbers yet.
• DSpine is strongest on code, math and tool calls. For prose, chat and image-heavy requests, DFlash2 or the target's built-in MTP (--spec-type draft-mtp --spec-draft-n-max 2, needs a target GGUF that includes the MTP layer) can be faster, especially on slower GPUs.
• Multiple GPUs: a two-device layer split (the default) was tested and gives the same output as one GPU. Qwen3.8 keeps a copy of its recurrent state for every drafted token, so if one card runs out of memory, lower --spec-draft-n-max or -c.
• Output can differ from runs without a drafter, even at temperature 0. Verification processes several tokens in one batch, which rounds slightly differently from one token at a time, so near-tied tokens can flip. Code and tool calls usually match; prose and image descriptions can diverge.

If you test on other hardware, please open a discussion with your numbers.

Benchmarks

All numbers were measured on the setup below:

Maximum context on 24 GB

For each setup, this is the largest -c that loads entirely on the GPU and serves requests with the q8_0 KV cache.

The limit comes from the target, not the drafter. Both drafters use a sliding-window KV cache (8k for DSpine), so their own cache stays small. Qwen3.8 is a hybrid model with recurrent layers, though. While it verifies a draft, it keeps a copy of its recurrent state for every drafted token, which comes to about 150 MB per copy. A 16-token DSpine block therefore costs about 1.2 GB more than an 8-token DFlash2 block, and that VRAM comes out of the context budget.

Speed at increasing context depth

These numbers come from llama-benchy, with --pp 0 --tg 256 --runs 1. The context is filled to each depth, and the table shows decode speed for 256 new tokens, in tok/s. Each setup ran with -c set to its maximum, and the last row for each setup is just below that maximum. A dash means the depth does not fit for that setup on 24 GB.

Default corpus (a Sherlock Holmes novel; the model writes prose)

Code corpus (llama.cpp C++ sources as context; the model mostly explains the code)

Prefill speed for the default corpus, in tok/s:

Each cell is a single run of 256 tokens. Prose and code explanations are the hardest content to draft, so expect noise of a few tok/s. On these corpora the drafters average only about 2.5-3.5 tokens per verify step.

Speed by task (short context)

This test used 10 chat prompts, 2 each of math, writing and summarize/translate, plus 4 code prompts in Python, C++, JavaScript and SQL. Each prompt generated up to 1,024 tokens with thinking disabled, about 6,000 tokens per setup. Temperature 1 used top_p 0.95, top_k 20 with probabilistic draft sampling.

DSpine Q4_K_M at temperature 0, by task:

Summary: On the RTX 3090, DSpine roughly doubles DFlash2's gain on code and math at short context, and it is still the fastest at every depth it fits. Its trade-off on a 24 GB card is context: about 64k at 15 drafted tokens, compared with about 100k for DFlash2 and 176k without speculation.

Convert it yourself

From the dspine branch:

python convert_hf_to_gguf.py /path/to/Qwen3.8-27B-DSpine-b16 \
  --target-model-dir /path/to/Qwen3.8-27B \
  --outtype bf16 --outfile Qwen3.8-27B-DSpine-b16-BF16.gguf

./build/bin/llama-quantize Qwen3.8-27B-DSpine-b16-BF16.gguf Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf Q4_K_M

--target-model-dir must point to the target model's config.json and tokenizer files. The target's weights are not needed.

License and credit

The weights are from tencent/Qwen3.8-27B-DSpine-b16, released under Apache-2.0. All credit for the model goes to the DSpine authors.

@article{zhang2026dspine,
  title   = {Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding},
  author  = {Zhang, Haohui and Chen, Keyu and Sun, Haocheng and Gu, Weibo and Qiao, Ruizhi and Sun, Xing and Jiang, Bo},
  journal = {arXiv preprint arXiv:2609.36173},
  year    = {2026}
}