K2-Horizon-7B-DSpark
A lossless DSpark draft model for IFM/K2-Horizon-7B, trained with speculators (training code).
- Up to 5.11× faster on math and 3.01× on code, 3.00× on average (temperature 0)
- Mean acceptance length 3.37 (block size 8)
Usage
vllm serve IFM/K2-Horizon-7B --port 8000 \
--trust-remote-code \
--reasoning-parser k2_horizon \
--max-model-len 32768 \
--speculative-config '{"model": "rasyosef/K2-Horizon-7B-DSpark", "num_speculative_tokens": 8}' \
--override-generation-config '{"temperature": 0}'
For tool use, add --enable-auto-tool-choice --tool-call-parser k2_horizon.
Details
- Qwen3-style, 4 layers, ~0.9B params, bfloat16, block size 8
- Hidden size 4096, MLP 8192, 32 query / 8 KV heads (head dim 128)
- First 3 layers sliding-window (512), last full attention
- Draft vocab 32,000 (from 250,624); aux hidden states from verifier layers 2/10/18/26/34
- 100,000 Open PerfectBlend prompts with verifier-generated responses, seq length 8192
- 3 epochs, lr 3e-4, cosine (4% warmup), loss
{"ce": 0.1, "tv": 0.9} - Trained on a single NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB)
Evaluation
RedHatAI/speculator_benchmarks, temperature 0, single A100, concurrency 1.
Throughput (tokens/s)
Mean output throughput. Baseline is K2-Horizon-7B without speculative decoding.
| subset | baseline (no drafter) | DSpark (this model) | speedup |
|---|---|---|---|
| math_reasoning | 71.4 | 364.5 | 5.11× |
| HumanEval | 71.0 | 213.5 | 3.01× |
| rag | 70.7 | 214.1 | 3.03× |
| qa | 70.8 | 217.6 | 3.07× |
| question | 71.2 | 176.9 | 2.48× |
| writing | 71.0 | 176.3 | 2.48× |
| translation | 71.2 | 190.9 | 2.68× |
| tool_call | 70.4 | 190.5 | 2.71× |
| summarization | 70.7 | 172.8 | 2.44× |
| average | 70.9 | 213.0 | 3.00× |
Acceptance at temperature 0
acceptance_length is mean tokens committed per verification step, including the bonus token (max 9.0 at block size 8).
| subset | acceptance_length | pos_0 | pos_1 | pos_2 | pos_3 | pos_4 | pos_5 | pos_6 | pos_7 |
|---|---|---|---|---|---|---|---|---|---|
| math_reasoning | 5.350 | 87.8% | 75.6% | 65.4% | 55.8% | 47.7% | 40.5% | 34.0% | 28.1% |
| HumanEval | 3.625 | 78.1% | 57.3% | 41.4% | 29.7% | 21.4% | 15.5% | 11.1% | 7.9% |
| rag | 3.411 | 75.4% | 54.5% | 38.6% | 26.8% | 18.7% | 12.4% | 8.6% | 6.0% |
| qa | 3.326 | 70.9% | 49.9% | 35.8% | 25.6% | 18.8% | 14.0% | 10.3% | 7.2% |
| question | 3.316 | 69.8% | 48.4% | 34.0% | 25.1% | 19.2% | 14.6% | 11.5% | 8.8% |
| writing | 3.284 | 69.3% | 47.9% | 34.0% | 25.0% | 18.9% | 14.3% | 11.0% | 7.9% |
| translation | 3.050 | 68.6% | 47.3% | 32.1% | 21.8% | 14.8% | 9.9% | 6.4% | 4.1% |
| tool_call | 3.018 | 72.0% | 48.5% | 32.0% | 20.4% | 12.9% | 8.0% | 5.0% | 3.0% |
| summarization | 2.863 | 68.6% | 44.5% | 28.9% | 18.9% | 11.8% | 7.1% | 4.1% | 2.4% |
Weighted across all subsets: 3.370 over 298,078 verification steps.
Limitations
Only works with K2-Horizon-7B. Sampling, concurrent load, and contexts beyond 8K tokens were not evaluated.
License
Apache 2.0