Qwen3-8B DSpark drafter, 5 layers, dense FFN 24576 (Setting 2, dim 24576, 1 epoch)
Speculative-decoding draft for Qwen/Qwen3-8B trained with SpecForge (DSpark). Dense 5-layer draft whose FFN width is 24576 (1.88 B parameters, all active); the dense counterpart of the MoE 8 x 24576 top-1 draft (same active FFN width per token). Attention 32 q / 8 KV heads x 128 with q/k norm; projector fc 20480->4096 over target layers 1/9/17/25/33; Markov head rank 256; block 16.
Training (1 epoch): perfectblend regen (temperature 0, no-think) prompts, 8192 context, qwen3-instruct template with no system prompt,
lr 6e-4 cosine over the single epoch with 4 % warm-up, 144 samples per optimizer step, 9,867 steps, Liger RMSNorm, four 4x GB300 nodes.
Code: https://github.com/yushengsu-thu/SpecForge/pull/22 (unchanged) + the draft config configs/qwen3-8b-5layer-dense-24576-dspark.json.
The checkpoint appears as 1ep/ when the epoch finishes. Serves with stock SGLang DSPARK (no MoE class needed).