English注册

drowzeys/DeepSeek-V4-Flash-DSpark-NVFP4-KV-1.5M-CTX-2xDGX-Spark

🤗 Hugging Face 来源text-generationapache-2.0-other⚠ 没有 SHA-256 权重今天更新
帮助这个模型通过 Pirate Face 分发

模型卡、文件列表和校验和已在此收录。如果你持有文件并有权分享，可以提交种子，让其他人从节点下载。

获取来源文件需要 Hugging Face 批准。
为此模型做种

DeepSeek-V4-Flash-DSpark · NVFP4-KV · 1.5M CTX · 3M Pool · C=12 · 2× DGX Spark

A serving recipe / configuration card for running deepseek-ai/DeepSeek-V4-Flash-DSpark with DSpark speculative decoding and an nvfp4_ds_mla 4-bit KV cache on 2× NVIDIA DGX Spark (GB10, sm_121a) at TP=2, for a 1.5M-token context window.

This card does not re-host model weights. NVFP4 here is a runtime KV-cache setting — the weights are the unmodified base model. Pull the base model from deepseek-ai and apply this recipe (below). Full sources, benchmark harness, and the vLLM 0.24.0 port fix live in the GitHub repo: https://github.com/drowzeys/keys-vLLm-0.24.0-Optimized-DeepSeekV4-Flash-DSpark-NVFP4-KV-1.5M-CTX-3M-Pool-C-12-on-2-DGX-Spark

Headline

  • 1.5M context window backed by a 3,231,736-token KV pool (util 0.85).
  • Coherent needle retrieval validated to 543,994 tokens — no garble.
  • C=12 concurrency at ~255 tok/s aggregate; 61–67% DSpark acceptance.
  • Runs on two DGX Spark boxes over RoCE/IB.

Standing config

Parameter Value
KV cache nvfp4_ds_mla (4-bit MLA KV)
Context window 1,500,000 (max_model_len)
KV pool 3,231,736 tokens (21.25 GiB @ util 0.85)
max_num_seqs 12
gpu_memory_utilization 0.85
Speculative DSpark, MTP_NUM_TOKENS=5
VLLM_USE_B12X_WO_PROJECTION 1 (high-acceptance-under-concurrency profile)
Topology 2× DGX Spark GB10, TP=2

Native trained context is 1,048,576; 1M–1.5M is RoPE-extended (VLLM_ALLOW_LONG_MAX_MODEL_LEN=1). Validated-coherent to 512k. See RESULTS.md.

Download & serve (two commands)

# 1) pull the (unmodified) base model
hf download deepseek-ai/DeepSeek-V4-Flash-DSpark --local-dir ./DeepSeek-V4-Flash-DSpark

# 2) build the GB10 serving image + launch with this recipe
#    (image build + launcher in the GitHub repo; run worker then head)
ssh <worker> '~/dspark-nvfp4-serve.sh 1'
~/dspark-nvfp4-serve.sh 0     # serves an OpenAI-compatible API on :8000

dspark-nvfp4-serve.sh (included here) carries the exact serve args, the NVFP4/DSpark/garble-fix env, and the 1.5M/util-0.85/seqs-12/WO=1 profile. Edit the fabric vars (MASTER, IF, HCA, GID) and model path for your cluster.

Files

  • dspark-nvfp4-serve.sh — the serving launcher (this exact config)
  • RESULTS.md — full benchmarks: concurrency, context, and long-context (6k→512k) sweeps
  • CREDITS.md — attribution

Credits

Special thanks to tonyd2wild (the nvfp4_ds_mla 1M stage recipe), MiaAI-Lab (DSpark DGX Spark packaging), Rafael Caricio (DSpark vLLM integration), and the vLLM project. Transplant kernels credited to aidendle94 (compiled flashinfer sparse-MLA + DeepGEMM), CosmicRaisins (Triton sparse-MLA), and hazyumps (GB10 indexer). Model: deepseek-ai/DeepSeek-V4-Flash-DSpark. Full details in CREDITS.md.

Support / Donations

If this work is useful, donations help fund more open GB10 / DGX Spark serving research:

  • Solana: drkeys.sol

Thank you 🙏