DeepSeek-V4-Flash-0731-REAP-DSpark

This is the complete 0xSero/DeepSeek-V4-Flash-0731-REAP checkpoint with its three DSpark MTP draft blocks (mtp.0/1/2) replaced by the fine-tuned head s3recap-p25-b0.1. It is meant for single-stream speculative decoding with the author's pure-CUDA engine on one NVIDIA Jetson AGX Thor. Measured there, the 8-prompt suite decodes at 28.38 tok/s, against 14.61 tok/s for plain autoregressive decode in the same run (1.94x).

The 43-layer backbone is unchanged. Shards 1–45, config.json, the tokenizer files, REAP_MANIFEST.json, reap_plan.json, LICENSE and model.safetensors.index.json are byte-identical to the base repo (same LFS SHA-256). Only shards 46–48, which hold the MTP blocks, differ.

Model overview

Which version should I use

Quickstart

The checkpoint has been run only with patrickbdevaney/deepseek-v4-flash-0731-cuda, a single-request engine that targets sm_110a. The commands below are that repo's documented launch path. The author ran them on a Jetson AGX Thor (CUDA 13.0) against a local checkpoint whose shards are byte-identical to this repo's.

huggingface-cli download patrickbdevaney/DeepSeek-V4-Flash-0731-REAP-DSpark --local-dir ~/models/DeepSeek-V4-Flash-0731-REAP-DSpark

git clone https://github.com/patrickbdevaney/deepseek-v4-flash-0731-cuda && cd deepseek-v4-flash-0731-cuda
bash scripts/build_server.sh                     # unit gates, server, terminal client
CKPT=~/models/DeepSeek-V4-Flash-0731-REAP-DSpark bash scripts/serve.sh   # OpenAI-compatible, port 8080
curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Write a binary search in Python."}]}'

Run nothing else that loads a model on the same machine: the weights occupy 100.4 GiB of the 122.8 GiB unified pool. Transformers, vLLM and SGLang have not been tested with this checkpoint. The base repo documents a vLLM preview smoke test on DGX Spark that needed a router patch for 160 experts, with DSpark disabled.

Evaluation

All measurements were taken on one Jetson AGX Thor with the weights as shipped. Logs are in the engine repo under evidence/.

Speculative decoding speed

The protocol is the engine's frozen 8-prompt suite, one prompt per category, with at least 200 generated tokens per prompt, greedy decoding and draft width 5. τ is the number of tokens committed per target forward. At width 5 it can reach 6 (five drafted tokens plus the correction token). Throughput is the suite mean.

• Against its own run's AR decode, the speed-up is 28.38 / 14.61 = 1.94x. Per-prompt ratios range from 1.44x (code_gen) to 2.33x (multi_turn).
• The stock head was 22.66 tok/s at width 6 on an earlier engine revision (HEAD_REGISTRY.md). The project quotes the move from 22.66 to 28.38 (+25.3%) as its end-to-end gain. That figure combines the head, the width change and engine work.
• On three copy-heavy probe prompts built for a suffix-drafting experiment, this head ran at 32.38, 31.38 and 34.97 tok/s (2.25–2.51x against 13.94 tok/s AR in that run; evidence/s6_probe.log).
• Speculation verifies greedily, so output is the same as plain greedy decode by construction. Each measurement run also checks the first 8 speculative tokens against base AR (LOSSLESS GATE ... PASS).

Acceptance by category (τ at width 5). The head was selected on the suite mean. That mean hides a trade:

Under the project's own release rule, this head fails 3 of 6 category floors: long_context, agentic_format and reasoning (docs/HEAD_REGISTRY.md). It was promoted and deployed on the mean. If your workload is mostly long-context reconstruction, the stock head may draft better.

Task accuracy

These scores measure the pruned checkpoint as served by this engine. They were taken with reasoning_effort=low, temperature 1.0 and top-p 0.95. Brackets are 95% intervals: Wilson for single-sample tasks, cluster bootstrap over problems for AIME and SciCode. A truncated item has no extractable answer and is scored wrong. The draft head does not change these numbers: the target weights are unchanged, and the acceptance rule draws every token from the target. Items were scored under more than one head (docs/EVALS.md).

• GPQA Diamond missed its pre-registered prediction. Before the 24K run, the project predicted 87–93% and set 85% as the falsification line. The run scored 83.8%. Of the 51 items that had hit the 8K cap, 19 still had not terminated at 24K, and the 32 that did terminate scored 78.1%. Items that finished inside 8K scored 93.9% (dev/archive/EVAL_NEXT_STEPS.md). No matched-harness run of the unpruned model exists, so the effect has not been attributed to pruning.
• Budget changes results. At 8K, GPQA Diamond reads 72.7% with 51 items truncated.
• AIME 2025 at 24K is incomplete (54 of 60 items) and is not reported.
• Not measured: the unpruned model under the same harness, long-context benchmarks, multi-turn agentic benchmarks, and any perplexity or agreement comparison against the unpruned model.

Hardware and memory

Everything was measured on one NVIDIA Jetson AGX Thor (sm_110a, 20 SMs, 122.8 GiB unified memory, CUDA 13.0, 240 GB/s measured streaming bandwidth). The server processes one request at a time.

The decode budget is dominated by bytes per token. The engine's roofline computes 11.20 GB read per decoded token. MLA projections account for 41.1% of that and routed experts for 30.8%. At 200 GB/s this gives a computed AR ceiling of 17.85 tok/s (docs/ROOFLINE.md). That ceiling is an estimate, not a measurement. Measured AR decode is 14.61 tok/s on the suite.

How it was made

1. Pruning, by 0xSero. REAP expert rankings were transferred from an observation run on the earlier deepseek-ai/DeepSeek-V4-Flash revision 60d8d707 (21,289 rows across coding, math, science, tool calling and agentic trajectories). Router alignment was checked against 0731: every old expert's nearest current row had the same ID, and the minimum layer-mean cosine was 0.9924. Then 160 of 256 experts were kept in all 46 MoE scopes, including the three MTP blocks. Details are in reap_plan.json, REAP_MANIFEST.json and the base card.
2. Draft-head fine-tune, by the author. Training covered the non-expert tensors of the three MTP blocks: 72 tensors, 517,318,215 parameters, covering attention, router, shared expert, hyper-connections, main_proj, Markov and confidence heads. Routed experts were copied untouched. The data was one epoch over 1,472 sequences, with 64 held out, generated by the base model itself from 1,536 prompts balanced over 8 categories; the agentic_format prompts came from BFCL. The loss mixed cross-entropy and total variation at 0.1/0.9, deficit weighting was clamped at 3.0, a KL anchor with β = 0.1 pulled toward the pre-training head, and the peak learning rate was 5e-5. Full recipe: docs/BEST_SETUP.md.
3. Install. The BF16 master was re-quantised to the checkpoint's own formats, FP8 E4M3 with freshly computed power-of-two E8M0 block scales, by tools/build_trained_head.py. That script refuses any tensor whose round-trip error exceeds 0.10. The result was written into shards 46–48: 2,977 MTP tensors, 7,010,423,612 bytes (6.53 GiB).

Limitations

• One machine, one engine. All speed and accuracy numbers come from one Jetson AGX Thor running the custom CUDA engine. No other runtime has been tested.
• Single stream. The engine has no batching across clients.
• Head is checkpoint-specific. It was trained against this 160-expert REAP and is not expected to transfer to another pruning or quantisation.
• Held-out drafting. Trained heads in this programme win the frozen suite but have lost to the untrained head on held-out continuation drafting against a paired control (wiki/draft-head-finetuning.md). The β anchor was added to address this. The effect is not shown to be eliminated.
• Pruned model. 37.5% of routed experts are removed and rankings were transferred, not re-observed on 0731. Accuracy versus the unpruned model is unmeasured under a matched harness.
• SHA256SUMS was replaced on 2026-11-06. The earlier file was the base repo's: it listed 21 files this repo does not contain and base-checkpoint hashes for shards 46–48. The new file covers every file in this repo except README.md (which changes with card edits), using the Hub's LFS SHA-256 for large files. Shards 46–48 match the engine's head_card.json (790139019c60…, 335d72f9624f…, 80f11aa77095…). Check a download with sha256sum -c SHA256SUMS.
• No chat-template code here. This repo omits the base repo's encoding/ directory. The CUDA engine has its own encoder; other runtimes should take encoding/ from the base repo.

Changelog and corrections

• 2026-08-27: first upload, with the full 48-shard checkpoint and the s3recap-p25-b0.1 head in shards 46–48.
• 2026-11-06: card rewritten and verified against repo files and build artifacts; a 2026-10-26 automated rewrite contained errors.

License and attribution

MIT, following the upstream and base repos. The DeepSeek copyright notice is retained in LICENSE.

• Upstream model: DeepSeek, deepseek-ai/DeepSeek-V4-Flash-0731, MIT.
• REAP pruning: 0xSero, 0xSero/DeepSeek-V4-Flash-0731-REAP, MIT.
• REAP method: Lasby et al., 2025, arXiv:2510.13999.
• This repo adds only the fine-tuned draft-head tensors in shards 46–48.

Citation

@misc{devaney2026dsv4dspark,
  title  = {DeepSeek-V4-Flash-0731-REAP-DSpark: a fine-tuned DSpark draft head for lossless speculative decoding on Jetson AGX Thor},
  author = {Devaney, Patrick},
  year   = {2026},
  url    = {https://huggingface.co/patrickbdevaney/DeepSeek-V4-Flash-0731-REAP-DSpark}
}