Erland/K2-Horizon-375B-A23B-DFlash-b8-v1

Model details

Training

• Framework: DeepSpec.
• Stage 1 : about 197,000 K2 answers at temperature 1.0, roughly 80% PerfectBlend prompts and 20% WildChat and glaive function-calling prompts
• Stage 2 : 54,464 greedy (temperature 0) K2 answers to production-style prompts, with reasoning effort high:
  • about 50% WildChat first turns;
  • about 27% multi-turn chats;
  • about 9% glaive function calling;
  • about 14% PerfectBlend.
• Hardware: 8x MI210 (training) with target hidden states extracted by vLLM on a second 8x MI210 node.
• Prompt sources and their licenses (prompts only; every answer was written by K2):
  • allenai/WildChat-1M, revision 7d6490e: ODC-BY (attribution required). Non-toxic, non-redacted conversations only, all languages; first turns, and the first 2-3 user turns with their real replies as context.
  • glaiveai/glaive-function-calling-v2, revision e7f4b64: Apache-2.0. First user turn, with the listed functions as tools.
  • mlabonne/open-perfectblend, revision af60f3c: Apache-2.0. First user turn, from the training split only.
  • the real-chat set used in stage 1.

Results

All numbers below use 64 test prompts (half production chat, half PerfectBlend), greedy decoding, 1,024 output tokens, and K2-Horizon-375B FP8 on 8x MI210 (TP8 / EP8). The speed-up is against the same server without a drafter.

3 guesses, combined build (see Serving):

Acceptance at 16 requests, 3 guesses:

That is 2.55 tokens per step on average, including the token K2 always adds.

Fixed-shape screen (16 requests unless noted; throughput against the drafter-free server):

Short answers gain little or lose a little: the drafter adds prefill work that a few decode steps cannot repay.

Serving

vllm serve <K2-Horizon-375B-A23B-FP8> \
  --tensor-parallel-size 8 --enable-expert-parallel \
  --speculative-config '{"method": "dflash", "model": "<this repo>", "num_speculative_tokens": 3,
                         "attention_backend": "TRITON_ATTN"}'

License

• The drafter weights are released under Apache-2.0, the license of the target model K2-Horizon-375B-A23B, whose answers and hidden states they were trained on.
• The DFlash method and its reference code (z-lab/dflash) are under the MIT license.

Citation

DFlash: Jian Chen, Yesheng Liang, Zhijian Liu. DFlash: Block Diffusion for Flash Speculative Decoding. ICML 2026, arXiv:2602.06036.

@misc{k2horizon2026,
  title  = {Introducing K2 Horizon: Frontier Performance, Radically Open},
  author = {{IFM Team}},
  year   = {2026},
  url    = {https://ifm.ai/blog/k2/},
}

@inproceedings{chen2026dflash,
  title     = {DFlash: Block Diffusion for Flash Speculative Decoding},
  author    = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2026},
  note      = {arXiv:2602.06036},
}