MMFlash draft model for speculative decoding of Qwen/Qwen3.8-27B on image and text inputs. It is not a standalone language model; it is paired with the target model in a speculative-decoding server (SGLang DFLASH).
Paper: MMFlash: Faster Speculative Decoding for Vision-Language Models.
Code: https://github.com/fscdc/MMFlash.
Project page: https://fscdc.github.io/MMFlash/.
MMFlash builds on DFlash block-diffusion drafting: the draft reads hidden states from five layers of the target and proposes a block of 16 tokens in one forward pass, and the target verifies the block in one forward pass. MMFlash keeps the DFlash architecture and is trained on multimodal data with a loss weighted by per-token visual-grounding (visual-KL) scores.
Qwen3.5-9B on a MathVista question, recorded in real time on one NVIDIA H200 with SGLang (greedy decoding).
[!TIP] Need a drafter for another model? Email us at fscnkucs@gmail.com and we can help train one. We will keep updating these releases, and a Qwen4 checkpoint is planned.
| Target model | Qwen/Qwen3.8-27B |
| Parameters | 1.73B (bf16) |
| Block size | 16 |
| Target hidden states | layers 1, 16, 31, 46, 61 (of 64) |
SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1 python -m sglang.launch_server \
--model-path Qwen/Qwen3.8-27B \
--speculative-algorithm DFLASH \
--speculative-draft-model-path FSCCS/Qwen3.8-27B-MMFlash \
--speculative-dflash-block-size 16 \
--attention-backend fa3 \
--mm-attention-backend sdpa \
--disable-radix-cache \
--max-running-requests 16 \
--reasoning-parser qwen3 \
--trust-remote-code
--max-running-requests 16 is required with --disable-radix-cache: SGLang then reserves about 2.3 GB of GDN verification states per allowed request at startup, and the default of 48 requests does not fit on one H200.
vllm serve Qwen/Qwen3.8-27B \
--dtype bfloat16 \
--max-model-len 32768 \
--attention-backend FLASH_ATTN \
--reasoning-parser qwen3 \
--speculative-config '{"method": "dflash", "model": "FSCCS/Qwen3.8-27B-MMFlash", "num_speculative_tokens": 15}'
Tested with vLLM 0.32.0, which serves DFlash drafts natively. num_speculative_tokens is the block size minus one.
Each cell is τ / speedup: τ is the average acceptance length (tokens accepted per verification step), and the speedup is the decoding throughput over Qwen3.8-27B without speculative decoding.
| Method | ChartQA | MMStar | DynaMath |
|---|---|---|---|
| MTP (15) | 6.36 / 2.47× | 5.25 / 2.16× | 6.63 / 2.68× |
| MMFlash (16) | 6.37 / 4.18× | 5.36 / 3.54× | 7.57 / 4.71× |
Values in parentheses are the number of draft steps for MTP and the block size for MMFlash. MTP is Qwen3.8-27B's built-in multi-token-prediction head.
MMFlash at 1, 8 and 32 requests in flight, on a single NVIDIA H200. Each cell is the decode throughput in tokens/s, with the speedup over Qwen3.8-27B alone at the same concurrency in parentheses.
| Concurrency | ChartQA | CharXiv | MMStar | MMBench | DynaMath | MathVista | MathVerse | Avg. |
|---|---|---|---|---|---|---|---|---|
| 1 | 277 (4.18×) | 286 (4.33×) | 234 (3.54×) | 209 (3.16×) | 311 (4.71×) | 278 (4.20×) | 296 (4.48×) | 270 (4.09×) |
| 8 | 762 (1.86×) | 907 (2.22×) | 927 (2.14×) | 606 (1.43×) | 1,300 (3.01×) | 1,032 (2.39×) | 1,169 (2.66×) | 958 (2.24×) |
| 32 | 932 (1.15×) | 1,320 (1.40×) | 1,211 (1.11×) | 946 (1.48×) | 1,627 (1.59×) | 1,405 (1.32×) | 1,628 (1.22×) | 1,295 (1.32×) |
For Qwen3.8-27B, concurrency 8 and 32 were served with --disable-radix-cache --mem-fraction-static 0.9. With the radix cache, SGLang keeps five state slots per request, and the MMFlash server held only 6 requests. At 32 in flight, the MMFlash server still holds at most 24 requests, because the GDN verification states take about 2.3 GB per request; the target alone holds 32.
@article{feng2026mmflash,
title = {MMFlash: Faster Speculative Decoding for Vision-Language Models},
author = {Feng, Sicheng and Chen, Zigeng and Wang, Song and Fang, Gongfan and Wang, Xinchao},
journal = {arXiv preprint arXiv:2611.18482},
year = {2026}
}
Qwen/Qwen3.8-27B