Tencent Hunyuan's DSpine drafter for Qwen3.8-27B, ready to run on Apple Silicon with dspine-hybrid-mlx.
DSpine is a parallel drafter. A small 5-layer model (2.0B parameters) proposes a block of tokens, and Qwen3.8-27B checks the whole block in one pass. You get the 27B model's own output, faster.
Qwen3.8-27B is a hybrid model, with 48 GatedDeltaNet (linear-attention) layers and 16 full-attention layers. Support for it, with exact rollback of the linear-attention state and the drafter's 8192-token sliding window, is now merged into Tencent's own dspine_mlx (Tencent/DSpine#3). You can run this drafter with either package and any MLX build of Qwen3.8-27B.
The weights are Tencent's bf16 checkpoint, unchanged (revision ea909c0), in the safetensors layout MLX loads directly.
pip install "git+https://github.com/internet-projects/dspine-hybrid-mlx.git"
dspine-hybrid-generate \
--target mlx-community/Qwen3.8-27B-4bit \
--draft netprojects/Qwen3.8-27B-DSpine-b16-MLX \
--prompt "Write a Python function that checks whether a number is prime." --baseline
--baseline also runs plain decoding and prints the speedup. Add --adaptive for the fastest greedy decoding with thinking off.
To use Tencent's package instead, clone Tencent/DSpine, run pip install -e dspine_mlx and pass this repo as the draft. dspine-hybrid-mlx remains the one-line install and adds --adaptive.
Greedy, compared with plain mlx-lm on the same target. Thinking on, 512 new tokens, 10 prompts (2 each from GSM8K, MATH-500, HumanEval, MBPP and MT-Bench), default settings (4-bit drafter):
| Target build | Plain mlx-lm | With DSpine | Speedup | Verify block |
|---|---|---|---|---|
4-bit (mlx-community/Qwen3.8-27B-4bit) |
31.9 tok/s | 64.7 tok/s | ×2.03 | 5 |
8-bit (mlx-community/Qwen3.8-27B-8bit)¹ |
17.9 tok/s | 52.1 tok/s | ×2.91 | 5 |
| bf16 | 9.1 tok/s | 39.1 tok/s | ×4.31 | 16 |
¹ Measured on a local conversion with the same settings as mlx-community/Qwen3.8-27B-8bit (8-bit, group 64, affine), made with mlx_lm.convert --hf-path Qwen/Qwen3.8-27B -q --q-bits 8 --q-group-size 64.
Thinking off: 4-bit 32.0 → 71.0 tok/s (×2.22), 8-bit 18.0 → 54.0 tok/s (×3.00), with --adaptive.
Long prompts (thinking off, 256 new tokens): with a 33k-token prompt, 4-bit 28.3 → 57.3 tok/s (×2.02) and 8-bit 16.7 → 47.7 tok/s (×2.85). Peak memory 29 GB and 42 GB.
Why block 5 and a 4-bit drafter. On an Apple GPU a quantized 27B forward pass costs about the same for 1 to 5 tokens and then steps up (4-bit: 34 ms for 1 token, 42 ms for 5, 89 ms for 16). Checking 5 drafted tokens per round accepts fewer of them than checking 16, but each round is much cheaper. The drafter runs once per round, and a 4-bit drafter proposes almost the same tokens as the bf16 one (3.40 vs 3.35 accepted per round). Use --block-size and --draft-bits (8, or 16 for bf16) to change them.
Acceptance and quality (thinking off, bf16 drafter, full 16-token block). These are not the conditions of the speed tables: short math answers reach about 7.2 tokens per round. Greedy, first 50 GSM8K test questions:
| Target build | Tokens per round | GSM8K accuracy |
|---|---|---|
| 4-bit | 7.25 | 90% |
| 8-bit | 7.32 | 84% |
| bf16 | 7.33 | 86% |
| Tencent's model card (GSM8K, T=0) | 7.20 |
The output is the target's own greedy output, up to bf16 rounding at exact near-ties. See the GitHub README for the checks.
| File | Notes |
|---|---|
model.safetensors |
DSpine drafter, bf16, 4.05 GB (Tencent revision ea909c0, unchanged) |
config.json |
Tencent's config: block 16, sliding window 8192, reads target layers 1/16/31/46/61; architectures is DSpineDraftModel, matching Tencent's naming (both names load) |
LICENSE, NOTICE |
Apache-2.0 |
Apache-2.0. Drafter by Tencent Hunyuan, for Qwen3.8-27B by Alibaba's Qwen team. MLX runtime built on Tencent DSpine and mlx-lm. Not affiliated with Tencent, Alibaba/Qwen or Apple.
Made by NetProjects.
tencent/Qwen3.8-27B-DSpine-b16