Qwen3.8-Flash-Next MTP draft head (Q8_0)

The multi-token-prediction head from Qwen/Qwen3.8-Flash-Next, 3.85 GiB at Q8_0.

Qwen trains it jointly with the model, so it drafts better than a separate small model would. Use it as the draft for any Qwen3.8-Flash-Next GGUF.

Setup

Works with upstream llama.cpp b11386 or newer (Qwen4Exp MTP support landed in ggml-org/llama.cpp#29761). No fork needed.

The target must be a GGUF that upstream llama.cpp loads, for example unsloth's Q8_0 or UD quants.

Updated 2026-10-05: the file now uses upstream's tensor names for the MTP head (blk.48.nextn.hc_head_*) and marks the MTP layer as a sparse-attention layer in the metadata, as upstream expects. The weights are unchanged. Older downloads fail on upstream with tensor 'blk.48.nextn.hc_head_norm.weight' not found: please download the file again.

Run

./build/bin/llama-server \
  -m Qwen3.8-Flash-Next.gguf \
  -md Qwen3.8-Flash-Next-MTP-Q8_0.gguf \
  -ngl 99 --n-gpu-layers-draft 99 \
  --spec-type draft-mtp --spec-draft-n-max 3 -c 32768

With -hf, add --mtp and llama-server picks up a sibling mtp- file from the same repo.

Adds ~3.9 GiB on top of the target.

Measurements

Radeon 8060S, 250 tokens at temp 0, warmed up first (measured before the rename; the weights are unchanged).

A 4-bit version of this head measured faster on both targets and is 1.6 GiB smaller.

Credits

Qwen3.8-Flash-Next support in llama.cpp is the work of Daniel Han (@danielhanchen), from ggml-org/llama.cpp#27742; the MTP graph is from #27739 (JJJYmmm). Base model by the Qwen team.

Quantized and published by Agention.

License

Qwen Community License 1.0, included as LICENSE.