Qwen3.5-2B-DSpark-GGUF
GGUF build of rasyosef/Qwen3.5-2B-DSpark for llama.cpp (DSpark speculative decoding is in mainline, ggml-org/llama.cpp #25173).
This is a standalone draft sidecar: it carries only the drafter (5 attention layers, rank-256 Markov head, confidence head, block size 8). Token embeddings and the LM head are shared from the target model at load time, so it must be paired with a Qwen3.5-2B GGUF target file, for example one from unsloth/Qwen3.5-2B-GGUF.
📦 Files
| file | quant | size | notes |
|---|---|---|---|
Qwen3.5-2B-DSpark-BF16.gguf |
BF16 | 914 MB | native training precision, recommended when memory allows |
Qwen3.5-2B-DSpark-F16.gguf |
F16 | 914 MB | same size as BF16, for backends without good BF16 support |
Qwen3.5-2B-DSpark-Q8_0.gguf |
Q8_0 | 491 MB | smallest file, for tighter memory budgets |
Choose the draft quant by memory budget. The target model quant is the main speed/quality lever and is independent of this file.
🏃 How to run (llama.cpp)
llama-server -hf unsloth/Qwen3.5-2B-GGUF:Q8_0 \
-hfd rasyosef/Qwen3.5-2B-DSpark-GGUF:Q8_0 \
--spec-type draft-dspark --spec-draft-n-max 8 --spec-draft-n-min 0 \
-fa on -ngl 99
The block size is read from the sidecar metadata (n-max is clamped to it). Speculative decoding is exact: the target verifies every proposed token, so greedy output equals the target alone; per-response timings report draft_n / draft_n_accepted.
Other models in the DSpark GGUF family:
📊 Acceptance and benchmarks
See rasyosef/Qwen3.5-2B-DSpark for acceptance-length metrics and training details.