DeepSeek-V4.1-Flash DSpark drafter, fine-tuned
A fine-tune of the three MTP layers (mtp.0 to mtp.2) that ship with DeepSeek-V4.1-Flash-1017 and drive its DSpark speculative decoding. Routed experts and embeddings are unchanged from the release. Everything else in the drafter was trained on the served model's own outputs.
The gain is small: more accepted draft tokens in non-thinking mode, about flat in thinking mode at temperature 1.
Files
| File | What it is |
|---|---|
mtp-trained.safetensors |
The 72 trained tensors in BF16/F32, release names (mtp.N.*). Attention, router gate and biases, shared experts, main_proj, norms, hyper-connection parameters, Markov head, confidence head. |
exl3/mtp_exl3_k5.safetensors |
The dense matmuls of the same weights in EXL3 at 5 bits (main_proj at 6), as served. mtp_exl3_k5.safetensors.json lists each tensor's relative quantization error (all under 7e-4). |
fp8/mtp-ft.safetensors |
Drop-in for the official checkpoint: all 2,401 mtp.* tensors in the release's own format. Routed experts are the release's bytes. The 25 dense matmuls are FP8 E4M3 with 32x32 blocks and power-of-two UE8M0 scales, quantized with the release's rule. Everything else is cast to the release's dtype. Speed gain untested (see below). |
fp8/model.safetensors.index.json |
The release index with every mtp.* entry pointed at mtp-ft.safetensors. |
fp8/fp8-error.json |
Per-tensor FP8 error against the trained weights, and how far each tensor moved from the release. |
Drop-in use: put fp8/mtp-ft.safetensors and fp8/model.safetensors.index.json into a copy of the official checkpoint, replacing its index. The original shards stay as they are; their mtp.* tensors are simply no longer referenced.
The FP8 quantizer was checked against the release itself: dequantizing each of its 25 FP8 drafter tensors and quantizing again gives the release's exact bytes in every block (551,424 blocks), except blocks whose maximum sits exactly on the rounding edge, where the values are still identical.
FP8 rounding costs up to 1.6% relative error per tensor, about the size of the fine-tune's own change (1.7% mean distance from the release). The live results below were measured with the EXL3 file, not the FP8 one, so the FP8 file's gain is not measured.
Results
Target: DeepSeek-V4.1-Flash-1017 in EXL3, tensor parallel over two DGX Spark (GB10), vLLM with DSpark, 5 draft tokens.
Held-out windows (1,635, not used in training), mean accepted tokens per step:
| Released | This fine-tune | |
|---|---|---|
| All | 4.404 | 4.548 (+3.3%) |
| Thinking mode, T=1 | +1.0% |
Live A/B, 48 agent turns not seen in training, same morning, same server build:
| Released | This fine-tune | |
|---|---|---|
| Non-thinking, T=0: accepted per step | 3.55 | 3.77 (+6%) |
| Non-thinking, T=0: tok/s at 1 / 8 users | 77.3 / 140.0 | 79.0 / 148.5 |
| Thinking, T=1: accepted per step | 1.85 to 1.89 | 1.91 to 1.97 (+3%) |
| Thinking, T=1: tok/s | flat |
Offline numbers run higher than live ones on this setup.
The larger win at temperature 1 is not in the weights. Set vLLM's draft_sample_method to probabilistic: the default greedy verifies argmax drafts as one-hot, so each draft token is accepted with the target's probability of the argmax. On the same server that change took thinking-mode decode from 59.7 to 67.9 tok/s at 1 user and from 114.3 to 126.8 at 8 users, with either drafter.
Training
- Init: the released
mtp.0tomtp.2. Routed experts and embeddings frozen. - Data: agent sessions (thinking and non-thinking) and Open-PerfectBlend prompts, regenerated by the served model. Target features (layers 37 to 39 and the LM-head input) captured from the served EXL3 model. Secrets scrubbed before training. The data is not published.
- One pass, 1,140 steps, learning rate 6e-5, micro batch 1 with 16 accumulation steps, sequence length 640, 256 anchors, block size 5.
- Loss: 0.1 cross-entropy + 0.9 L1, position decay gamma 4. Markov head rank 256, confidence head trained jointly.
- Two RTX PRO 6000, one run per GPU.
One pitfall: if your server uses sliding-window bounded replay for the decoder, captured features for the early part of each prompt can come back as zeros. Check captures for all-zero rows before training on them.
License
MIT, same as the base model.