Qwen 3.8 Flash Next - MTP Draft Speculative Model (GGUF)

This repository provides working MTP (Multi-Token Prediction) draft models in GGUF format for Qwen 3.8 Flash Next (and experimental architectures with hyper-connections / QSA / Hadamard KV rotations), powered by cafe-llama.cpp.

Compatible Engine

This MTP model requires the fork implementation supporting hyper-connection hidden states, QSA Hadamard KV rotation, and MTP layer graph generation:

Repository: https://github.com/quimmedes/cafe-llama.cpp

git clone https://github.com/quimmedes/cafe-llama.cpp.git

Building from Source

NVIDIA CUDA (Windows / Linux)

CMake configure with CUDA backend

cmake -B build -DGGML_CUDA=ON

Build Release

cmake --build build --config Release -j 2 2. Vulkan (Cross-Platform AMD / Intel / NVIDIA)

Requires Vulkan SDK installed

cmake -B build -DGGML_VULKAN=ON cmake --build build --config Release -j 2 3. AMD ROCm / HIP (Linux / Windows) cmake -B build -DGGML_HIP=ON -DAMDGPU_TARGETS="gfx1100;gfx1030" cmake --build build --config Release -j 2 4. Apple Metal (macOS) cmake -B build -DGGML_METAL=ON cmake --build build --config Release -j 2 5. CPU Only (AVX2 / AVX-512) cmake -B build -DGGML_CUDA=OFF -DGGML_VULKAN=OFF cmake --build build --config Release -j 2 -j N : The number of CPU threads the processor will use to compile, a safe number is the amount of physical cores of the processor.

Available Files

mtp-Qwen3.8-Flash-Next-Q4_K_M.gguf: Quantized Q4_K_M MTP draft model (~2.65 GB) - Recommended
mtp-Qwen3.8-Flash-Next-Q6_K.gguf: Quantized Q6_K MTP draft model (~3.24 GB)
mtp-Qwen3.8-Flash-Next-Q8_0.gguf: Quantized Q8_0 MTP draft model (~3.94 GB)
mtp-Qwen3.8-Flash-Next-BF16.gguf: Full precision BF16 MTP draft model (~7.40 GB)

Usage with llama.cpp / llama-server

Run llama-server or llama-cli with --spec-type draft-mtp and point -md to your preferred MTP file. Setting --spec-draft-n-max 2 provides optimal acceptance rate (~50-70%) for a 1-layer MTP draft head:

llama-server   -m Qwen3.8-Flash-Next-UD-IQ3_XXS-00001-of-00003.gguf   -md mtp-Qwen3.8-Flash-Next-Q4_K_M.gguf   --spec-type draft-mtp   --spec-draft-n-max 2   -ngl 999   -hmoe   -fa on   -ctk q8_0 -ctv q8_0 -kvu   -c 8192 -b 1024 -ub 128

CLI Example:

llama-cli   -m Qwen3.8-Flash-Next-UD-IQ3_XXS-00001-of-00003.gguf   -md mtp-Qwen3.8-Flash-Next-Q4_K_M.gguf   --spec-type draft-mtp   --spec-draft-n-max 2   -ngl 999   -hmoe   -fa on   -ctk q8_0 -ctv q8_0   -p "Explain quantum entanglement simply."

quimmedes/Qwen3.8-Flash-Next-MTP-GGUF