Spark-X2.6-1.7B-DSpark-GGUF

Spark-X2.6-1.7B-DSpark-GGUF provides a BF16 GGUF draft model for speculative decoding with Spark-X2.6-1.7B-GGUF. The target model can use BF16 or quantized GGUF weights.

DSpark accelerates generation through speculative decoding: the draft proposes up to seven tokens per drafting pass, and the target model verifies the proposed tokens in parallel, reducing the number of sequential target decoding steps.

Model Details

Property	Spark-X2.6-1.7B-DSpark
Target model	Spark-X2.6-1.7B
Draft parameters	345M
Draft weight precision	BF16
Draft layers	5 SWA layers
Sliding-window size	512
Hidden size	2048
Draft tokens per forward pass	7
Target hidden-state layers	`[2, 8, 14, 19, 25]`

Performance

Mean Acceptance Length

We evaluate acceptance with a draft length of 7 tokens and a maximum output length of 4096 tokens. Acceptance length includes one additional target token, giving a maximum of 8.

Table notation:

T: target model temperature.
High / Low / No thinking: target model thinking modes.
Prob. / Greedy: probabilistic sampling / argmax draft decoding.

Dataset	T=1, High, Prob.	T=1, Low, Prob.	T=1, No thinking, Prob.	T=0, High, Greedy
AIME25	3.5443	3.5919	4.4806	4.1771
GSM8K	3.7875	4.0283	4.4305	4.4367
MATH500	3.9027	3.9425	4.7694	4.5103
HumanEval	3.5324	3.5795	4.0170	4.4917
MBPP	3.6004	3.7508	3.8723	4.3375
Arena-Hard-v2	3.0905	3.0219	2.9768	3.9412
Average	3.5763	3.6525	4.0911	4.3158

On-Device Inference

We compare inference throughput with and without DSpark on NVIDIA RTX 4090 (llama.cpp) and Apple M5 Pro (MLX), using BF16 and quantized target models (Q4_K_M on RTX 4090 and INT4 on M5 Pro).

Dataset	BF16 · RTX 4090	BF16 · M5 Pro	Q4_K_M · RTX 4090	INT4 · M5 Pro
AIME25	2.03× 220 → 447 tok/s	1.80× 68 → 123 tok/s	1.44× 424 → 612 tok/s	1.15× 140 → 161 tok/s
GSM8K	2.43× 220 → 536 tok/s	1.97× 68 → 134 tok/s	1.81× 424 → 769 tok/s	1.25× 137 → 172 tok/s
MATH500	2.40× 220 → 530 tok/s	2.04× 69 → 140 tok/s	1.71× 424 → 724 tok/s	1.30× 135 → 176 tok/s
HumanEval	1.96× 220 → 430 tok/s	1.85× 68 → 126 tok/s	1.49× 422 → 628 tok/s	1.18× 139 → 164 tok/s
MBPP	1.96× 221 → 434 tok/s	1.93× 68 → 132 tok/s	1.43× 426 → 609 tok/s	1.23× 140 → 172 tok/s
Arena-Hard-v2	1.83× 219 → 401 tok/s	1.66× 68 → 114 tok/s	1.33× 422 → 560 tok/s	1.06× 137 → 146 tok/s
Average	2.10× 220 → 463 tok/s	1.88× 68 → 128 tok/s	1.53× 424 → 650 tok/s	1.20× 138 → 165 tok/s

Usage with llama.cpp

Use XHToken/llama.cpp to run Spark DSpark and send requests through an OpenAI-compatible API. The following commands are for Linux, Bash, and NVIDIA GPUs.

Branch: master (22c0ae1871b60123c5afc008a78fe9c6fffa2858). Integration into upstream llama.cpp is in progress.

1. Build

Clone the source and build:

git clone --branch master https://github.com/XHToken/llama.cpp.git
cd llama.cpp
cmake -S . -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build-cuda --target llama-server -j8

2. Start the Server

set -o pipefail
./build-cuda/bin/llama-server \
  -m models/Spark-target.gguf \
  -md models/Spark-draft.gguf \
  --alias spark-dspark \
  --spec-type draft-dspark \
  --spec-draft-sampling greedy \
  --spec-draft-n-max 7 \
  --spec-draft-p-min 0 \
  -ngl 999 -ngld 999 \
  -ctk f16 -ctv f16 \
  -ctkd f16 -ctvd f16 \
  -fa on \
  -c 8192 -np 1 \
  -b 2048 -ub 512 -t 8 \
  --jinja \
  --host 127.0.0.1 --port 8080 \
  2>&1 | tee dspark-server.log

3. Send an Example Request

curl -fsS http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "spark-dspark",
    "messages": [
      {
        "role": "user",
        "content": "你好"
      }
    ],
    "temperature": 1.0,
    "top_p": 0.95,
    "max_tokens": 512,
    "stream": false
  }'

License

The Spark-X2.6 model series is licensed under the Apache 2.0 License.

Citation

If you find our work helpful, feel free to give us a cite.

@misc{sparkx2.6,
    title  = {Spark-X2.6-4B&1.7B: Advancing the Frontier of On-Device Agents for Real-World Tasks},
    author = {SparkLLM Team},
    year   = {2026}
}

XHToken/Spark-X2.6-1.7B-DSpark-GGUF