TandemLLM drafter for Qwen3.8-27B, block 16

This is the DFlash2 block drafter that TandemLLM serves for Qwen3.8-27B at block 16. One pass of it guesses 15 tokens, and the target model checks all of them in one verify pass. It is one of the two fine-tuned drafters published with the paper StairCut: sizing speculative draft trees on a measured verification-cost staircase (Khaled Bakeer, 2026; Zenodo: 10.5281/zenodo.23045275). The paper's PDF is in the NVFP4 repository. Its twin is 0xBakeer/TandemLLM-Qwen3.8-27B-DFlash2-b8.

Intended use

Speculative decoding of Qwen/Qwen3.8-27B in TandemLLM. It is not a standalone language model: it proposes tokens, and the target decides which ones are written. In the paper's served configuration, StairCut drafts with this one every round and cuts its tree to the size the measured verify cost favours.

What changed from the base drafter

We fine-tuned the five draft layers of z-lab/Qwen3.8-27B-DFlash2 on 2026-09-17. The target model is unchanged. The training data was sequences the target wrote itself plus public text that the target labelled. The loss was cross-entropy on the target's argmax plus a KL term on its top 64 tokens. We trained with AdamW at a learning rate of 3e-5 for 500 steps at block 16.

The drafter has no embedding and no output head of its own. It borrows both from the target, so it works with any of TandemLLM's weight profiles: FP8, balanced and NVFP4 (0xBakeer/TandemLLM-Qwen3.8-27B-NVFP4).

Speed

The paper measured it on one DGX Spark with the published NVFP4 weights, one request at a time, on the benchmark row serve-single-i256-o256-v1 (256 prompt tokens, 256 new tokens, 50 requests, greedy, persistent lookup store off). StairCut with this drafter reads 49.89 tok/s. The same drafter at its fixed width (a node budget of 16, rising to 24 after 32 committed tokens, plus a chain of up to 32 rows after long accepted paths) reads 44.80 tok/s, and plain greedy decoding 13.69 tok/s.

Files

How to use with TandemLLM

hf download 0xBakeer/TandemLLM-Qwen3.8-27B-DFlash2-b8 --local-dir ~/tandem/ft-b8
hf download 0xBakeer/TandemLLM-Qwen3.8-27B-DFlash2-b16 --local-dir ~/tandem/ft-b16

In ops/serve.env of the TandemLLM repository, set CKPT8=~/tandem/ft-b8 and CKPT16=~/tandem/ft-b16, then run bash ops/start.sh. The server passes them as --dflash2-ckpt and --dflash2-ckpt16. docs/speculative-decoding.md in that repository says how the engine loads and routes them.

License and attribution

Apache-2.0. The base drafter is DFlash2 for Qwen3.8-27B by Inco AI (incoai/Qwen3.8-27B-DFlash2, mirrored as z-lab/Qwen3.8-27B-DFlash2), released under Apache-2.0. The target is Qwen/Qwen3.8-27B by the Qwen team, also Apache-2.0. The change from the base drafter is the fine-tune described above. Keep this attribution and the license when you pass the weights on.

Citation

@misc{bakeer2026staircut,
  title  = {StairCut: sizing speculative draft trees on a measured verification-cost staircase},
  author = {Bakeer, Khaled},
  year   = {2026},
  publisher = {Zenodo},
  doi    = {10.5281/zenodo.23045275},
  url    = {https://doi.org/10.5281/zenodo.23045275}
}