Qwen3.6-35B-A3B-DSPARK-GGUF

GGUF conversion of DSpark speculative-decoding draft model for llama.cpp.

This is a DSPARK draft model, not a standalone language model. It must be used together with a compatible Qwen3.6-35B-A3B target model.

Model

• Base model: Koopah/Qwen3.6-35B-A3B-NVFP4-DSPARK
• Target model: Qwen/Qwen3.6-35B-A3B
• Format: GGUF
• Quantization: BF16

Status

Support for DSpark-format speculators is being added to llama.cpp in ggml-org/llama.cpp#26275. Until merged, use a build compiled from the PR branch.

Compatibility

The target model and speculator must use compatible tokenizer, vocabulary and architecture metadata. A mismatch can cause loading errors or incorrect output.

Qwen3.6-35B-A3B Speculative Decoding Benchmark

This repository contains benchmark results for speculative decoding on Qwen3.6-35B-A3B, comparing:

• No speculative decoding
• DSpark with n_max = 2
• DSpark with n_max = 3
• DSpark with n_max = 5
• DFlash with n_max = 7

The benchmark covers code generation, structured output, reasoning, factual QA, translation, creative writing, and repetitive low-entropy output.

These results are specific to the tested model, quantization, llama.cpp branch/build, hardware configuration, and benchmark prompts.
Acceptance rate alone is not sufficient to predict throughput; draft cost, accepted depth, and the number of speculative rounds also matter.

---

Summary

Recommended settings For Consumer GPU

Main findings

• DFlash max 7 is the strongest overall speculative-decoding configuration.
• DSpark max 5 is slower than the no-speculation baseline over the full benchmark.
• Between DSpark max 2 and max 3, max 3 is the better default for code and structured tasks.
• DSpark max 2 reduces some of the slowdown on explanation, translation, and long code review, but those workloads are still generally faster without speculative decoding.
• Fixed long draft lengths work very well on predictable output, but waste substantial computation on high-entropy text.

---

Overall comparison

Aggregate wall-clock results

Effective throughput is calculated as:

total predicted tokens / total wall-clock time

Relative to no speculative decoding

DFlash max 7 completes the benchmark approximately:

• 18% faster than the no-speculation baseline
• 31% faster than DSpark max 5

---

DSpark max 2 vs max 3

Per-task throughput

Aggregate comparison

The aggregate difference is very small:

• max 3 is only about 0.6% faster overall
• max 3 produces 535 more draft tokens
• max 3 accepts 112 more draft tokens

This means the third draft position is close to the overall break-even point.

Recommendation

Use:

--spec-draft-n-max 3

as the default DSpark setting when the workload is mainly:

• source code
• code completion
• JSON
• structured output
• repetitive output
• constrained reasoning

Use max 2 only when reducing the penalty on less predictable natural-language output is more important.

However, translation, creative writing, and long free-form explanations are still usually better with speculative decoding disabled.

---

DSpark max 5 vs DFlash max 7

Per-task throughput

Acceptance comparison

DFlash generates slightly fewer total draft tokens than DSpark max 5, while accepting substantially more:

This suggests that DFlash is not winning by brute-force drafting. Its proposals are better aligned with the target model, resulting in:

• deeper accepted runs
• fewer speculative rounds
• fewer wasted proposals
• better utilization of longer draft sequences

---

Workload analysis

Code and structured output

DFlash max 7 is clearly the strongest configuration.

These tasks are predictable enough for long speculative runs to be accepted frequently.

In particular:

• code_completion reaches 94.3% acceptance
• repeat_pattern reaches 97.1% acceptance
• json_output reaches 73.7% acceptance

This is the ideal operating region for speculative decoding.

Factual QA and reasoning

DFlash remains useful on several medium-entropy tasks:

DSpark max 5 does not reach the same break-even point on these workloads.

Translation and creative writing

Both DSpark and DFlash perform poorly here.

Acceptance falls to approximately 11-14%, so most later draft positions are wasted.

For these workloads, use:

No speculative decoding

Explanation and long code review

Long explanatory text and code-review prose also remain difficult:

Although the subject may be technical, the output itself is less constrained than code completion. The next token distribution remains too broad for a fixed seven-token draft chain to be consistently useful.

---

Why aggregate acceptance rate is not enough

A single aggregate acceptance percentage hides positional behavior.

For example, two configurations may have similar aggregate acceptance while behaving very differently:

• one may frequently accept three or four tokens per round
• another may accept only the first token in many rounds
• one may perform fewer total speculative rounds
• another may repeatedly compute later draft positions that are rejected

The most useful additional metric would be positional acceptance:

P1, P2, P3, P4, P5, P6, P7

This would expose the acceptance-decay knee directly.

Expected behavior from these results:

• DFlash likely maintains stronger acceptance at later positions on code and JSON.
• DSpark likely decays earlier, especially after the second or third position.
• Translation and creative writing likely collapse immediately after the first position.

---

Dynamic confidence truncation

A fixed n_max applies the same draft depth to every speculative round, regardless of task entropy.

A better implementation would stop draft generation dynamically when confidence becomes too low:

if confidence[position] < threshold:
    stop drafting

This could allow:

• code completion to use all seven positions
• JSON to use five to seven positions
• factual QA to stop after two to four positions
• translation and creative output to stop after one position
• very uncertain rounds to skip speculative decoding entirely

Dynamic early exit could preserve the large DFlash gains on structured workloads while reducing severe regressions on free-form text.

---

Raw benchmark results

DSpark max 2

code_python        pred= 192 draft= 160 acc= 110 rate=0.688 tok/s=138.0
code_cpp           pred=  60 draft=  52 acc=  35 rate=0.673 tok/s=133.6
explain_concept    pred= 192 draft= 220 acc=  80 rate=0.364 tok/s=105.3
summarize          pred=  51 draft=  52 acc=  24 rate=0.462 tok/s=114.8
qa_factual         pred= 192 draft= 188 acc=  96 rate=0.511 tok/s=122.4
translation        pred=  17 draft=  20 acc=   6 rate=0.300 tok/s=89.0
creative_short     pred=  36 draft=  50 acc=  10 rate=0.200 tok/s=85.3
stepwise_math      pred= 192 draft= 173 acc= 103 rate=0.595 tok/s=129.5
json_output        pred= 192 draft= 141 acc= 120 rate=0.851 tok/s=155.3
long_reasoning     pred= 192 draft= 204 acc=  89 rate=0.436 tok/s=112.6
repeat_pattern     pred= 192 draft= 131 acc= 125 rate=0.954 tok/s=168.2
code_completion    pred= 192 draft= 157 acc= 112 rate=0.713 tok/s=140.6
long_code_review   pred= 192 draft= 220 acc=  80 rate=0.364 tok/s=101.8

{
  "n_requests": 13,
  "total_predicted": 1892,
  "total_draft": 1768,
  "total_draft_accepted": 990,
  "aggregate_accept_rate": 0.56,
  "wall_s_total": 17.43
}

DSpark max 3

code_python        pred= 192 draft= 195 acc= 126 rate=0.646 tok/s=148.8
code_cpp           pred=  54 draft=  54 acc=  37 rate=0.685 tok/s=150.8
explain_concept    pred= 192 draft= 330 acc=  79 rate=0.239 tok/s=90.0
summarize          pred=  52 draft=  63 acc=  30 rate=0.476 tok/s=120.9
qa_factual         pred= 192 draft= 240 acc= 110 rate=0.458 tok/s=126.3
translation        pred=  17 draft=  30 acc=   6 rate=0.200 tok/s=82.6
creative_short     pred=  38 draft=  66 acc=  15 rate=0.227 tok/s=90.5
stepwise_math      pred= 192 draft= 222 acc= 116 rate=0.522 tok/s=133.0
json_output        pred= 192 draft= 175 acc= 132 rate=0.754 tok/s=164.3
long_reasoning     pred= 192 draft= 264 acc= 102 rate=0.386 tok/s=111.8
repeat_pattern     pred= 192 draft= 147 acc= 141 rate=0.959 tok/s=195.6
code_completion    pred= 192 draft= 187 acc= 128 rate=0.684 tok/s=154.5
long_code_review   pred= 192 draft= 330 acc=  80 rate=0.242 tok/s=90.7

{
  "n_requests": 13,
  "total_predicted": 1889,
  "total_draft": 2303,
  "total_draft_accepted": 1102,
  "aggregate_accept_rate": 0.4785,
  "wall_s_total": 17.3
}

DSpark max 5

code_python        pred= 192 draft= 293 acc= 132 rate=0.451 tok/s=133.0
code_cpp           pred=  54 draft=  65 acc=  42 rate=0.646 tok/s=172.5
explain_concept    pred= 192 draft= 498 acc=  89 rate=0.179 tok/s=82.5
summarize          pred=  51 draft=  95 acc=  31 rate=0.326 tok/s=113.9
qa_factual         pred= 192 draft= 375 acc= 115 rate=0.307 tok/s=112.2
translation        pred=  17 draft=  50 acc=   7 rate=0.140 tok/s=71.1
creative_short     pred=  35 draft= 110 acc=  12 rate=0.109 tok/s=70.5
stepwise_math      pred= 192 draft= 311 acc= 128 rate=0.412 tok/s=131.4
json_output        pred= 192 draft= 226 acc= 145 rate=0.642 tok/s=175.7
long_reasoning     pred= 192 draft= 370 acc= 116 rate=0.314 tok/s=113.2
repeat_pattern     pred= 192 draft= 170 acc= 157 rate=0.923 tok/s=239.9
code_completion    pred= 192 draft= 224 acc= 146 rate=0.652 tok/s=178.8
long_code_review   pred= 192 draft= 500 acc=  89 rate=0.178 tok/s=82.3

{
  "n_requests": 13,
  "total_predicted": 1885,
  "total_draft": 3287,
  "total_draft_accepted": 1209,
  "aggregate_accept_rate": 0.3678,
  "wall_s_total": 18.39
}

DFlash max 7

code_python        pred= 192 draft= 239 acc= 156 rate=0.653 tok/s=206.1
code_cpp           pred=  54 draft=  63 acc=  45 rate=0.714 tok/s=229.5
explain_concept    pred= 192 draft= 552 acc= 110 rate=0.199 tok/s=98.5
summarize          pred=  51 draft=  98 acc=  36 rate=0.367 tok/s=147.9
qa_factual         pred= 192 draft= 371 acc= 137 rate=0.369 tok/s=146.3
translation        pred=  17 draft=  70 acc=   8 rate=0.114 tok/s=68.8
creative_short     pred=  35 draft= 140 acc=  16 rate=0.114 tok/s=73.7
stepwise_math      pred= 192 draft= 279 acc= 150 rate=0.538 tok/s=184.3
json_output        pred= 192 draft= 217 acc= 160 rate=0.737 tok/s=237.4
long_reasoning     pred= 192 draft= 322 acc= 145 rate=0.450 tok/s=173.2
repeat_pattern     pred= 192 draft= 171 acc= 166 rate=0.971 tok/s=300.9
code_completion    pred= 192 draft= 175 acc= 165 rate=0.943 tok/s=275.9
long_code_review   pred= 192 draft= 546 acc= 111 rate=0.203 tok/s=96.8

{
  "n_requests": 13,
  "total_predicted": 1885,
  "total_draft": 3243,
  "total_draft_accepted": 1405,
  "aggregate_accept_rate": 0.4332,
  "wall_s_total": 14.08
}

No speculative decoding

code_python        pred= 192 draft=   0 acc=   0 rate=n/a tok/s=128.8
code_cpp           pred=  54 draft=   0 acc=   0 rate=n/a tok/s=129.6
explain_concept    pred= 192 draft=   0 acc=   0 rate=n/a tok/s=129.6
summarize          pred=  51 draft=   0 acc=   0 rate=n/a tok/s=128.6
qa_factual         pred= 192 draft=   0 acc=   0 rate=n/a tok/s=129.1
translation        pred=  17 draft=   0 acc=   0 rate=n/a tok/s=126.3
creative_short     pred=  40 draft=   0 acc=   0 rate=n/a tok/s=128.7
stepwise_math      pred= 192 draft=   0 acc=   0 rate=n/a tok/s=130.9
json_output        pred= 192 draft=   0 acc=   0 rate=n/a tok/s=129.5
long_reasoning     pred= 192 draft=   0 acc=   0 rate=n/a tok/s=127.4
repeat_pattern     pred= 192 draft=   0 acc=   0 rate=n/a tok/s=129.3
code_completion    pred= 192 draft=   0 acc=   0 rate=n/a tok/s=125.3
long_code_review   pred= 192 draft=   0 acc=   0 rate=n/a tok/s=126.5

{
  "n_requests": 13,
  "total_predicted": 1890,
  "total_draft": 0,
  "total_draft_accepted": 0,
  "aggregate_accept_rate": null,
  "wall_s_total": 16.69
}

---

Final recommendation

For a single general DSpark preset:

--spec-draft-n-max 3

For maximum performance on code, JSON, agent output, and other predictable workloads:

DFlash with --spec-draft-n-max 7

For translation, creative writing, roleplay, and long free-form explanations:

Disable speculative decoding

The benchmark strongly suggests that the ideal long-term solution is not a single fixed draft length, but confidence-based dynamic draft truncation.