DeepSeek-V4-Flash-0731 DSpark Graph-8 on four GB10 nodes

A reproducible four-node NVIDIA GB10 deployment and evidence package for the official deepseek-ai/DeepSeek-V4-Flash-0731 checkpoint. The validated stack uses TP4, DSpark K5 speculative decoding, NVFP4 DS-MLA KV cache, FlashInfer B12X MoE, dual-rail RoCE v2, and ordinary CUDA Graph capture.

Independent community recipe. This is not an official DeepSeek release and contains no model weights. Obtain the official checkpoint from DeepSeek-AI. DSpark accelerates target-verified generation; it does not change the target model's knowledge or semantics.

Headline result

The promoted profile was selected from matched four-node tests:

TP4 / 4 GB10 nodes
DSpark K5
max sequences: 8
max batched tokens: 16,384
max model length: 1,048,576

Matched controls:

• C8 aggregate versus the DSpark C4 incumbent: +37.13%
• C8 aggregate versus identical no-spec C8: +20.12%
• Single-stream decode versus identical no-spec C8: +81.64%
• Identical no-spec C8 aggregate: 154.06 tok/s

These are workload-specific, warm, bounded-output measurements. They are not a universal throughput floor and should not be compared across different prompts, tokenizers, output lengths, or throughput definitions.

Why K5/C8/16K was promoted

A K4/C8 candidate reached 192.22 tok/s aggregate, but regressed code by 11.45% and prose by 5.14%. K5/C8 with a 32K batched-token budget reached 183.91 tok/s and had 5.108 s p95 TTFT. The promoted K5/C8/16K profile gave the strongest balanced result.

Qualification gates

• Official checkpoint revision: 9e165c30e2704aec5d9d593cce3eebd58bbef1cb
• 76-file checkpoint manifest verified independently on all four nodes
• Runtime image ID: sha256:3430d6614a8e2925f34d059af6caf05aff42387326db4d05639a60f10f2654d8
• Model identity and exact basic completion: passed
• Closed-world strict JSON: passed
• Native tool call and tool-result continuation: passed
• Fresh uncached long-context retrieval: 134,028 prompt tokens, exact result
• Configured context: 1,048,576 tokens
• Observed KV capacity after restart: approximately 4.80 million tokens
• Coordinated four-rank stop/restart: passed
• C8 benchmark correctness gate: passed
• All eight RoCE HCAs carried traffic; captured error-counter deltas were zero
• Busy-aware watchdog passed during a fresh 131,230-token request
• Final containers: running, zero restarts, not OOM-killed

Runtime topology

Use four nodes with ranks 0..3. Rank 0 is the API head; ranks 1–3 launch with HEADLESS=1. Every node must have a complete verified checkpoint at the same path and its own .env.node derived from runtime/.env.example.

world size / tensor parallel: 4
pipeline parallel: 1
DSpark draft tokens: 5
KV cache: nvfp4_ds_mla
MoE backend: flashinfer_b12x
VLLM_USE_BREAKABLE_CUDAGRAPH=0
max sequences: 8
max batched tokens: 16384
max model length: 1048576

The nofile limit of 1,048,576 is mandatory. Without it, the first four-node launch failed during NCCL all-gather with Too many open files.

Quick start

1. Verify the official checkpoint and pinned runtime image on all four nodes.
2. Copy runtime/.env.example to a private .env.node on each node.
3. Set unique NODE_RANK, VLLM_HOST_IP, fabric interface/HCA values, and rank-0 MASTER_ADDR.
4. Start ranks 1–3 first, then rank 0:

COMPOSE_DISABLE_ENV_FILE=1 docker compose --env-file .env.node   -f runtime/docker-compose.yml -p deepseek-v4-0731-4xgb10 up -d

Keep the API loopback-only unless you deliberately provide authentication, rate limits, and firewalling through a reverse proxy.

Verify

python3 scripts/production_proof.py   --base-url http://127.0.0.1:8000   --model deepseek-v4-flash-0731-4xgb10   --context-tokens 160000

python3 scripts/benchmark.py   --base-url http://127.0.0.1:8000   --model deepseek-v4-flash-0731-4xgb10   --repetitions 3 --output benchmark.json

python3 scripts/benchmark_concurrency.py   --base-url http://127.0.0.1:8000   --model deepseek-v4-flash-0731-4xgb10   --concurrency 8 --rounds 3 --output c8.json

Repository contents

• runtime/docker-compose.yml — four-node TP4/DSpark runtime
• runtime/.env.example — node-local configuration template
• scripts/production_proof.py — model, tool, JSON, and bounded context gates
• scripts/benchmark.py — workload-specific streaming benchmark
• scripts/benchmark_concurrency.py — reproducible C8 aggregate benchmark
• benchmarks/production-summary.json — compact matched-campaign evidence
• benchmarks/publication-smoke.json — live pre-publication proof, workload smoke, and C8 smoke
• docs/METHODOLOGY.md — definitions, controls, caveats, and rejected variants

License and attribution

Package code and documentation are MIT-licensed. Model weights and all runtime dependencies remain governed by their upstream licenses and terms.

• CREDITS.md — contributor roles, direct runtime dependencies, recipe lineage, comparative references, and source links
• NOTICE — pinned deployed artifacts and concise third-party notice

This is an independent community recipe, not an official DeepSeek, vLLM, NVIDIA, Anemll, or other upstream release or endorsement.