DeepSeek-V4-Flash-DSpark · NVFP4-KV · 1.5M CTX · 3M Pool · C=12 · 2× DGX Spark
A serving recipe / configuration card for running
deepseek-ai/DeepSeek-V4-Flash-DSpark
with DSpark speculative decoding and an nvfp4_ds_mla 4-bit KV cache on 2× NVIDIA
DGX Spark (GB10, sm_121a) at TP=2, for a 1.5M-token context window.
This card does not re-host model weights. NVFP4 here is a runtime KV-cache setting — the weights are the unmodified base model. Pull the base model from
deepseek-aiand apply this recipe (below). Full sources, benchmark harness, and the vLLM 0.24.0 port fix live in the GitHub repo: https://github.com/drowzeys/keys-vLLm-0.24.0-Optimized-DeepSeekV4-Flash-DSpark-NVFP4-KV-1.5M-CTX-3M-Pool-C-12-on-2-DGX-Spark
Headline
- 1.5M context window backed by a 3,231,736-token KV pool (util 0.85).
- Coherent needle retrieval validated to 543,994 tokens — no garble.
- C=12 concurrency at ~255 tok/s aggregate; 61–67% DSpark acceptance.
- Runs on two DGX Spark boxes over RoCE/IB.
Standing config
| Parameter | Value |
|---|---|
| KV cache | nvfp4_ds_mla (4-bit MLA KV) |
| Context window | 1,500,000 (max_model_len) |
| KV pool | 3,231,736 tokens (21.25 GiB @ util 0.85) |
max_num_seqs |
12 |
gpu_memory_utilization |
0.85 |
| Speculative | DSpark, MTP_NUM_TOKENS=5 |
VLLM_USE_B12X_WO_PROJECTION |
1 (high-acceptance-under-concurrency profile) |
| Topology | 2× DGX Spark GB10, TP=2 |
Native trained context is 1,048,576; 1M–1.5M is RoPE-extended
(VLLM_ALLOW_LONG_MAX_MODEL_LEN=1). Validated-coherent to 512k. See RESULTS.md.
Download & serve (two commands)
# 1) pull the (unmodified) base model
hf download deepseek-ai/DeepSeek-V4-Flash-DSpark --local-dir ./DeepSeek-V4-Flash-DSpark
# 2) build the GB10 serving image + launch with this recipe
# (image build + launcher in the GitHub repo; run worker then head)
ssh <worker> '~/dspark-nvfp4-serve.sh 1'
~/dspark-nvfp4-serve.sh 0 # serves an OpenAI-compatible API on :8000
dspark-nvfp4-serve.sh (included here) carries the exact serve args, the NVFP4/DSpark/garble-fix
env, and the 1.5M/util-0.85/seqs-12/WO=1 profile. Edit the fabric vars (MASTER, IF, HCA, GID)
and model path for your cluster.
Files
dspark-nvfp4-serve.sh— the serving launcher (this exact config)RESULTS.md— full benchmarks: concurrency, context, and long-context (6k→512k) sweepsCREDITS.md— attribution
Credits
Special thanks to tonyd2wild (the nvfp4_ds_mla 1M stage recipe), MiaAI-Lab (DSpark
DGX Spark packaging), Rafael Caricio (DSpark vLLM integration), and the vLLM project.
Transplant kernels credited to aidendle94 (compiled flashinfer sparse-MLA + DeepGEMM),
CosmicRaisins (Triton sparse-MLA), and hazyumps (GB10 indexer). Model:
deepseek-ai/DeepSeek-V4-Flash-DSpark. Full details in CREDITS.md.
Support / Donations
If this work is useful, donations help fund more open GB10 / DGX Spark serving research:
- Solana:
drkeys.sol
Thank you 🙏