DCOCO drafter for Qwen/Qwen3-8B

This repository contains the DCOCO (DFlash Contextual Corrector) speculative-decoding drafter for Qwen/Qwen3-8B, from the paper Contextual Correction of Parallel Drafts for Speculative Decoding. It is a draft model: serve it together with Qwen/Qwen3-8B, which verifies every token, so the output distribution is exactly the target model's.

Paper: Contextual Correction of Parallel Drafts for Speculative Decoding
Code: github.com/two-delta/DCOCO

How it works

Diffusion-style drafters such as DFlash propose a whole block of tokens in a single pass, but each position is predicted without seeing which tokens were chosen before it. DCOCO keeps the top-16 candidates at every drafted position and scores how well each candidate fits each candidate at the next position, using candidate representations and a pair state over adjacent slots, all in one small batched pass. A prefix-first walk over those scores then picks the draft. This restores intra-block dependencies at a cost of a few percent of the drafting cycle.

Model details

Usage

DCOCO runs in vLLM, through the speculative-decoding dflash method; vLLM selects the corrector from the checkpoint's DCOCODraftModel architecture. This requires a vLLM build with DCOCO support, such as the DCOCO image (vllm/vllm-openai:v0.31.0 with the DCOCO overlay).

vllm serve Qwen/Qwen3-8B \
  --speculative-config '{"method": "dflash", "model": "TD-Labs/Qwen3-8B-DCOCO", "num_speculative_tokens": 15}'

Check that the server log shows Resolved architecture: DCOCODraftModel. With any other architecture, vLLM serves the drafter as plain DFlash without the corrector. Then query it like any OpenAI-compatible server:

curl localhost:8000/v1/chat/completions -H 'content-type: application/json' -d '{
  "model": "Qwen/Qwen3-8B",
  "messages": [{"role": "user", "content": "How many positive whole-number divisors does 196 have?"}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

num_speculative_tokens can be lowered (down to 1) but not raised above 15.

Results

All results below are from the paper (Tables 1, 7, 8, 9 and 19). Setup: one NVIDIA B200 GPU, vLLM 0.27, concurrency 1 unless noted, thinking off with at most 2,048 generated tokens (4,096 with thinking on), 128 prompts per benchmark (all 30 of AIME-25, all 80 of MT-Bench, 880 of SPEED-Bench). Throughput (tok/s) is the median per-request rate (output tokens / request latency, including prefill); speedup is relative to the target model alone. τ is the mean accepted length per verification round, including the bonus token. T=0 is a single greedy run; T=1 is the mean over seeds 42, 43 and 44.

Per-benchmark results (thinking off)

Greedy decoding (T=0) in the first four columns, sampling (T=1) in the last four.

Comparison with other parallel drafters (average over the ten benchmarks)

Thinking off:

Thinking on (extended reasoning):

Throughput under concurrent requests (thinking off)

Aggregate output tokens/s, averaged over the ten benchmarks.

Limitations

• The drafter is trained for Qwen/Qwen3-8B and must be served with exactly that target model.
• It needs the target's full vocabulary (no reduced draft vocabulary).
• It was trained on thinking-off generations; the speedup is lower with extended thinking (see above).
• The speedup depends on the task: highest on math and code, lower on open-ended chat.

Citation

@misc{eisenstadt2026contextualcorrectionparalleldrafts,
      title={Contextual Correction of Parallel Drafts for Speculative Decoding}, 
      author={Roy Eisenstadt and Harel Lidar and Tal Gonen and Omer Luxembourg and Itamar Zimerman and Lior Wolf and Eli David and Ilan Naiman},
      year={2026},
      eprint={2610.12570},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2610.12570}, 
}