Qwen3.6-27B-DFlash-GGUF

GGUF conversion of z-lab/Qwen3.6-27B-DFlash for llama.cpp.

This is a DFlash draft model, not a standalone language model. It must be used together with a compatible Qwen3.6-27B target model.

Model

• Base model: z-lab/Qwen3.6-27B-DFlash
• Target model: Qwen/Qwen3.6-27B
• Format: GGUF
• Quantization: Q4_K_M

Compatibility

Requires a recent version of llama.cpp with DFlash support.

Tested with:

• llama.cpp b9831 or newer

Usage

Example:

llama-server \
  -m Qwen3.6-27B.gguf \
  -md Qwen3.6-27B-DFlash-Q4_K_M.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 4

or

llama-cli \
  -m Qwen3.6-27B.gguf \
  -md Qwen3.6-27B-DFlash-Q4_K_M.gguf \
  --spec-type draft-dflash

Conversion

Converted from the original Hugging Face model using the latest convert_hf_to_gguf.py.

No model weights were modified.

Benchmark (RTX 5070 Ti)

Benchmarked using mtp-bench.

max 2

python mtp-bench.py
  code_python        pred= 192 draft= 134 acc= 123 rate=0.918 tok/s=59.2
  code_cpp           pred=  54 draft=  36 acc=  36 rate=1.000 tok/s=62.6
  explain_concept    pred= 192 draft= 172 acc= 105 rate=0.611 tok/s=47.2
  summarize          pred=  46 draft=  32 acc=  29 rate=0.906 tok/s=59.9
  qa_factual         pred= 183 draft= 146 acc= 109 rate=0.747 tok/s=52.9
  translation        pred=  17 draft=  16 acc=   8 rate=0.500 tok/s=43.1
  creative_short     pred=  36 draft=  38 acc=  17 rate=0.447 tok/s=39.3
  stepwise_math      pred= 192 draft= 141 acc= 119 rate=0.844 tok/s=56.3
  json_output        pred= 192 draft= 130 acc= 125 rate=0.962 tok/s=60.6
  long_reasoning     pred= 192 draft= 148 acc= 117 rate=0.790 tok/s=54.6
  repeat_pattern     pred= 192 draft= 127 acc= 127 rate=1.000 tok/s=62.5
  code_completion    pred= 191 draft= 128 acc= 127 rate=0.992 tok/s=62.3
  long_code_review   pred= 192 draft= 186 acc=  97 rate=0.521 tok/s=43.2

Aggregate: {
  "n_requests": 13,
  "total_predicted": 1871,
  "total_draft": 1434,
  "total_draft_accepted": 1139,
  "aggregate_accept_rate": 0.7943,
  "wall_s_total": 39.62
}

max 3

python mtp-bench.py
  code_python        pred= 192 draft= 162 acc= 137 rate=0.846 tok/s=66.7
  code_cpp           pred=  54 draft=  45 acc=  40 rate=0.889 tok/s=66.7
  explain_concept    pred= 192 draft= 226 acc= 115 rate=0.509 tok/s=48.4
  summarize          pred=  46 draft=  42 acc=  33 rate=0.786 tok/s=61.3
  qa_factual         pred= 192 draft= 199 acc= 124 rate=0.623 tok/s=54.6
  translation        pred=  17 draft=  24 acc=   9 rate=0.375 tok/s=38.8
  creative_short     pred=  36 draft=  48 acc=  20 rate=0.417 tok/s=41.7
  stepwise_math      pred= 192 draft= 175 acc= 131 rate=0.749 tok/s=61.1
  json_output        pred= 192 draft= 150 acc= 141 rate=0.940 tok/s=71.6
  long_reasoning     pred= 192 draft= 171 acc= 134 rate=0.784 tok/s=63.6
  repeat_pattern     pred= 192 draft= 143 acc= 143 rate=1.000 tok/s=75.6
  code_completion    pred= 191 draft= 145 acc= 142 rate=0.979 tok/s=72.9
  long_code_review   pred= 192 draft= 234 acc= 112 rate=0.479 tok/s=46.4

Aggregate: {
  "n_requests": 13,
  "total_predicted": 1880,
  "total_draft": 1764,
  "total_draft_accepted": 1281,
  "aggregate_accept_rate": 0.7262,
  "wall_s_total": 35.39
}

max 4

python mtp-bench.py
  code_python        pred= 192 draft= 183 acc= 145 rate=0.792 tok/s=70.7
  code_cpp           pred=  54 draft=  48 acc=  43 rate=0.896 tok/s=78.5
  explain_concept    pred= 192 draft= 281 acc= 120 rate=0.427 tok/s=48.5
  summarize          pred=  46 draft=  48 acc=  34 rate=0.708 tok/s=67.3
  qa_factual         pred= 171 draft= 208 acc= 119 rate=0.572 tok/s=58.4
  translation        pred=  17 draft=  28 acc=  11 rate=0.393 tok/s=42.1
  creative_short     pred=  36 draft=  64 acc=  21 rate=0.328 tok/s=39.6
  stepwise_math      pred= 192 draft= 190 acc= 142 rate=0.747 tok/s=69.3
  json_output        pred= 192 draft= 160 acc= 150 rate=0.938 tok/s=82.8
  long_reasoning     pred= 192 draft= 209 acc= 138 rate=0.660 tok/s=64.3
  repeat_pattern     pred= 192 draft= 152 acc= 152 rate=1.000 tok/s=86.7
  code_completion    pred= 191 draft= 156 acc= 151 rate=0.968 tok/s=86.2
  long_code_review   pred= 192 draft= 280 acc= 120 rate=0.429 tok/s=48.0

Aggregate: {
  "n_requests": 13,
  "total_predicted": 1859,
  "total_draft": 2007,
  "total_draft_accepted": 1346,
  "aggregate_accept_rate": 0.6707,
  "wall_s_total": 33.39
}

max 5

python mtp-bench.py
  code_python        pred= 192 draft= 203 acc= 150 rate=0.739 tok/s=72.7
  code_cpp           pred=  54 draft=  55 acc=  44 rate=0.800 tok/s=77.3
  explain_concept    pred= 192 draft= 345 acc= 121 rate=0.351 tok/s=44.0
  summarize          pred=  46 draft=  55 acc=  36 rate=0.654 tok/s=65.5
  qa_factual         pred= 171 draft= 265 acc= 119 rate=0.449 tok/s=51.6
  translation        pred=  17 draft=  35 acc=  11 rate=0.314 tok/s=37.6
  creative_short     pred=  36 draft=  80 acc=  21 rate=0.263 tok/s=35.8
  stepwise_math      pred= 192 draft= 221 acc= 145 rate=0.656 tok/s=67.3
  json_output        pred= 192 draft= 178 acc= 155 rate=0.871 tok/s=83.9
  long_reasoning     pred= 192 draft= 231 acc= 144 rate=0.623 tok/s=65.1
  repeat_pattern     pred= 192 draft= 160 acc= 158 rate=0.988 tok/s=93.1
  code_completion    pred= 191 draft= 165 acc= 157 rate=0.952 tok/s=90.6
  long_code_review   pred= 192 draft= 325 acc= 126 rate=0.388 tok/s=47.0

Aggregate: {
  "n_requests": 13,
  "total_predicted": 1859,
  "total_draft": 2318,
  "total_draft_accepted": 1387,
  "aggregate_accept_rate": 0.5984,
  "wall_s_total": 33.41
}

Observations

• n_max = 2 provides the highest acceptance rate.
• n_max = 4 provides the best overall throughput.
• Increasing to n_max = 5 provides almost no additional speed while noticeably reducing acceptance rate.

Notes

This repository contains only the DFlash draft model.

A compatible Qwen3.6-27B GGUF target model is required for speculative decoding.

Credits

• z-lab — Original DFlash model
• Qwen Team — Qwen3.6-27B
• ggml-org/llama.cpp — GGUF format and DFlash inference implementation

License

This repository contains a converted GGUF version of the original DFlash draft model.

All original licenses, usage restrictions, and intellectual property remain with the upstream authors. Please refer to the original repositories for complete licensing information.