TandemLLM drafter for Qwen3.8-27B, block 8

This is the DFlash2 block drafter that TandemLLM serves for Qwen3.8-27B at block 8. One pass of it guesses 7 tokens, and the target model checks all of them in one verify pass. It is one of the two fine-tuned drafters published with the paper StairCut: sizing speculative draft trees on a measured verification-cost staircase (Khaled Bakeer, 2026; Zenodo: 10.5281/zenodo.23045275). The paper's PDF is in the NVFP4 repository. Its twin is 0xBakeer/TandemLLM-Qwen3.8-27B-DFlash2-b16.

Intended use

Speculative decoding of Qwen/Qwen3.8-27B in TandemLLM. It is not a standalone language model: it proposes tokens, and the target decides which ones are written. In the paper's served configuration, StairCut runs the block 16 twin and releases this one at the first round; this drafter at its fixed width, with a node budget of 16, is the paper's block 8 baseline.

What changed from the base drafter

We fine-tuned the five draft layers of z-lab/Qwen3.8-27B-DFlash2 on 2026-09-17. The target model is unchanged. The training data was sequences the target wrote itself plus public text that the target labelled. The loss was cross-entropy on the target's argmax plus a KL term on its top 64 tokens. We trained with AdamW at a learning rate of 3e-5 for 500 steps at block 8.

The drafter has no embedding and no output head of its own. It borrows both from the target, so it works with any of TandemLLM's weight profiles: FP8, balanced and NVFP4 (0xBakeer/TandemLLM-Qwen3.8-27B-NVFP4).

Speed

The paper measured it on one DGX Spark with the published NVFP4 weights, one request at a time, on the benchmark row serve-single-i256-o256-v1 (256 prompt tokens, 256 new tokens, 50 requests, greedy, persistent lookup store off). This drafter alone at its fixed width reads 47.13 tok/s; StairCut with the block 16 twin reads 49.89 tok/s, and plain greedy decoding 13.69 tok/s.

Files

How to use with TandemLLM

hf download 0xBakeer/TandemLLM-Qwen3.8-27B-DFlash2-b8 --local-dir ~/tandem/ft-b8
hf download 0xBakeer/TandemLLM-Qwen3.8-27B-DFlash2-b16 --local-dir ~/tandem/ft-b16

In ops/serve.env of the TandemLLM repository, set CKPT8=~/tandem/ft-b8 and CKPT16=~/tandem/ft-b16, then run bash ops/start.sh. The server passes them as --dflash2-ckpt and --dflash2-ckpt16. docs/speculative-decoding.md in that repository says how the engine loads and routes them.

License and attribution

Apache-2.0. The base drafter is DFlash2 for Qwen3.8-27B by Inco AI (incoai/Qwen3.8-27B-DFlash2, mirrored as z-lab/Qwen3.8-27B-DFlash2), released under Apache-2.0. The target is Qwen/Qwen3.8-27B by the Qwen team, also Apache-2.0. The change from the base drafter is the fine-tune described above. Keep this attribution and the license when you pass the weights on.

Citation

@misc{bakeer2026staircut,
  title  = {StairCut: sizing speculative draft trees on a measured verification-cost staircase},
  author = {Bakeer, Khaled},
  year   = {2026},
  publisher = {Zenodo},
  doi    = {10.5281/zenodo.23045275},
  url    = {https://doi.org/10.5281/zenodo.23045275}
}