KAT-Coder-V2.5-Dev-DFlash2-2.6B-A0.3B

A Mixture-of-Experts DFlash2 drafter for Kwaipilot/KAT-Coder-V2.5-Dev.

Modern MoE target models are well suited to expert offloading on memory-constrained systems and large-scale expert parallelism at datacenter scale. Applying the same sparse structure to speculative drafters is therefore a natural direction. Strong precedents include DeepSeek-V3 MTP and DeepSeek-V4-Flash-0731 DSpark.

This release explores the same direction with a DFlash2 drafter: approximately 2.6B total parameters with approximately 0.3B activated per token.

Results

Setting	Value
Engine	vLLM
Hardware	NVIDIA H200, TP1
Concurrency	1
Target	`Kwaipilot/KAT-Coder-V2.5-Dev` BF16
Thinking sampling	`temperature=1.0`, `top_p=0.95`, `top_k=20`, `presence_penalty=1.5`
Thinking mode	`enable_thinking=true`
Non-thinking sampling	`temperature=0.7`, `top_p=0.8`, `top_k=20`, `presence_penalty=1.5`
Non-thinking mode	`enable_thinking=false`
Max new tokens	4,096
Context	native 262,144
Draft tokens	7 per verification step

Benchmark	Samples	Thinking AL	Non-thinking AL
AA-LCR	100	2.8870	3.3894
AIME26	30	2.8959	4.1591
GSM8K	1,319	4.2453	5.2930
HumanEval	164	4.0568	5.9836
LiveCodeBench	1,055	2.7090	3.5688
LongBench-v2	402	2.4484	2.8255
MATH500	500	3.6183	4.7679
MBPP	257	3.6543	4.5800
MT-Bench	80	2.5998	2.8009
SWE-bench Pro	731	3.1425	3.4428
SpeedBench / coding	80	3.3456	4.0887
SpeedBench / humanities	80	2.2735	2.7807
SpeedBench / math	80	2.4900	3.3639
SpeedBench / multilingual	80	3.0767	3.0240
SpeedBench / qa	80	2.5138	2.4313
SpeedBench / rag	80	3.3953	3.5579
SpeedBench / reasoning	80	2.8214	3.1751
SpeedBench / roleplay	80	1.8487	1.8848
SpeedBench / stem	80	2.3774	3.0061
SpeedBench / summarization	80	3.0177	2.6967
SpeedBench / writing	80	2.2223	2.3711

Accepted length is calculated from the raw server counters:

accepted_length = 1 + accepted_tokens / draft_calls

Architecture

Setting	Value
Draft layers	3
Auxiliary hidden-state IDs	`[3, 12, 21, 30, 39]`
Hidden size	2,048
MoE FFN	256 routed experts, top-8 routing
Routed expert intermediate	512
Shared expert	1 expert, intermediate size 512
Router	softmax, top-k normalization
Shared-expert gate	sigmoid
Attention	32 Q heads / 4 KV heads
Head dimension	128
Attention window	sliding window 2,048
Draft attention	non-causal inside the draft block
RoPE theta	1e7
Norm / activation	RMSNorm eps 1e-6 / SiLU
Dynamic convolution	kernel 2, group 16
Candidate selector	rank 256, top-k 16
Block	7 draft tokens, `sample_from_anchor=true`
Draft vocabulary	248,320
Parameters	2,638,551,808 total / 298,130,176 activated per token

Example: Run with vLLM

This is a drafter checkpoint, not a standalone language model. Pair it with KAT-Coder or a compatible quantized version:

vllm serve Kwaipilot/KAT-Coder-V2.5-Dev \
  --tensor-parallel-size 1 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --language-model-only \
  --speculative-config '{
    "method": "dflash",
    "model": "DaoCloud/KAT-Coder-V2.5-Dev-DFlash2-2.6B-A0.3B",
    "num_speculative_tokens": 7
  }'

This checkpoint currently requires a vLLM build containing both vLLM PR #54154, which adds the sample_from_anchor=true DFlash query layout, and vLLM PR #54475, which adds MoE DFlash2.

Checkpoint

The BF16 checkpoint contains 63 tensors and 2,638,551,808 parameters.

File	SHA256
`model.safetensors`	`8e96fdfa924405443b566be3a961eb087cd6991665040b05d7c3ec739d5469ee`

Future work

Adaptive verification: explore confidence-head-driven adaptive draft budgets for high-concurrency serving.

Acknowledgements

This work builds on Kwaipilot/KAT-Coder-V2.5-Dev, vllm-project/speculators, and vllm-project/vllm.

License

Apache-2.0.

DaoCloud/KAT-Coder-V2.5-Dev-DFlash2-2.6B-A0.3B