A self-contained, 512K-qualified DeepSeek-V4.1-Flash deployment for one 128 GB NVIDIA DGX Spark.
On an ASUS GX10, this fixed K154 release handled a 524,293-token prompt, returned all five buried values, and generated 44 more tokens. Warm short coding prompts decoded at a 27.79 tok/s median with 1.32 s median TTFT.
The released DeepSeek-V4.1-Flash checkpoint is about 510 GB—far beyond the memory of one Spark. Khaled Bakeer (0xBakeer) built and published the single-Spark runtime that made this possible: Engram lookups from NVMe, a resident expert arena, CB3 experts, Triton kernels, DSpark speculative decoding, and the benchmark-driven pruning path.
This release takes the next deployment step. It permanently packages 154 routed experts in each of the 40 main layers, removes the original main routed-expert tensors, and verifies every retained shard before serving. There is no routing trace to supply, no machine-local warm-start artifact, and no expert re-quantization at startup.
This repository contains the complete self-contained model payload: retained DeepSeek weights, the 40-layer K154 CB3-v2 expert pack, tokenizer, encoding, configuration, DSpark drafter, manifest, and digests. The matching runtime is
sayyidfareed/deepseek-v4.1-flash-next-dgx-spark-512k, tagv1.0.0.
| Test | Result |
|---|---|
| Engine capacity | 1,048,576 tokens |
| Validated request | 524,293 prompt + 44 output = 524,337 tokens |
| Distributed needle retrieval | 5/5 at 5%, 25%, 50%, 75%, 95% |
| Long-context prefill | 147.69 tok/s |
| Long-context decode | 5.69 tok/s |
| Median decode, five coding languages | 27.794 tok/s |
| Minimum warm decode | 25.235 tok/s |
| Maximum warm decode | 33.438 tok/s |
| Median warm TTFT | 1.324 s |
| HumanEval | 150/164 (91.5%) |
| HumanEval+ Mini | 142/164 (86.6%) |
| Executable coding microbenchmark | 29/34 |
| Minimum available memory during 512K prefill | 7.09 GiB |
| New swap used during 512K request | 1.17 GiB |
Hardware: NVIDIA GX10 / GB10 with 121.63 GiB usable unified memory and local NVMe. The model service was the only large workload on the box. Short-speed numbers are the second identical five-language pass after graph warmup; the first pass contained one first-shape compilation outlier and is retained in the qualification evidence rather than silently mixed into the warm result.
DeepSeek has 384 routed expert blocks in each of 40 main layers. For every token, the router normally picks six. The complete checkpoint contains 15,360 of those expert blocks, which is too much to keep in a Spark's unified memory.
K154 means this release keeps a fixed set of 154 experts per layer: 6,160 total. Those experts are stored in a compact three-bit-per-row CB3-v2 format and loaded into the 89.16 GB resident arena. The router is restricted to that exact set, so serving does not fall back to missing source experts.
This is not a retrained model. Dense weights, Engram tables, tokenizer, configuration, and DSpark drafter come from DeepSeek's checkpoint. The tradeoff is deliberate: fixed-expert pruning makes a 748B-class model practical on one 128 GB machine, but it can change quality outside the coding and reasoning workloads used to choose and validate the set.
The original checkpoint's two Engram tables account for about 203 GB and stay on NVMe. The routed experts account for another 288.78 GB. Keeping the original routed tensors alongside a resident packed copy would make a supposedly publishable model hundreds of gigabytes larger than necessary.
This repository therefore contains:
The 40 rewritten base shards and 40 CB3 layer shards are authenticated by the manifest. Startup fails before the API port binds if a shard, tensor layout, selection, or source identity differs.
Requirements:
hf download sayyidfareed/DeepSeek-V4.1-Flash-Next-DGX-Spark-512K \
--local-dir /models/DeepSeek-V4.1-Flash-Next-DGX-Spark-512K
git clone --branch v1.0.0 \
https://github.com/sayyidfareed/deepseek-v4.1-flash-next-dgx-spark-512k.git
cd deepseek-v4.1-flash-next-dgx-spark-512k
MODEL_PARENT=/models docker compose -f compose.k154.yaml up --build
The server is ready when the log says serving DeepSeek-V4.1-Flash-Next-DGX-Spark-512K. It exposes an OpenAI-compatible API
on 127.0.0.1:8000 by default:
curl http://127.0.0.1:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "DeepSeek-V4.1-Flash-Next-DGX-Spark-512K",
"messages": [{"role":"user","content":"Write a lock-free ring buffer in Rust."}],
"temperature": 0,
"max_tokens": 1024,
"stream": true,
"tool_choice": "none"
}'
The API has no built-in authentication. Keep it on loopback or place an authenticated proxy and trusted-network boundary in front of it.
| Setting | Value |
|---|---|
| Fixed experts | 154 per layer × 40 layers = 6,160 |
| Expert format | CB3-v2, three bits per row |
| Resident arena | 89.16 GB |
| Transient slots | 8 |
| Free-memory floor | 2.5 GiB |
| Maximum sequences | 1 |
| Engine capacity | 1,048,576 tokens |
| Prefill chunk | 256 tokens |
| Sparse-window replay | enabled |
| DSpark speculative decoding | enabled |
| Dense FP4 groups | attention and wo_a |
| Head format | FP8 |
The immutable launcher supplies these values and refuses routing-trace or arbitrary extra flags. Changing the runtime, expert set, memory floor, or kernels creates a new profile that needs fresh qualification.
The qualified container used Ubuntu 24.04, Python 3.12.3, CUDA 13.0, PyTorch 2.13.0+cu130, Triton 3.7.1, Transformers 5.17.0, Tokenizers 0.23.2, Safetensors 0.8.0, NumPy 2.3.5, and SymPy 1.14.0. The Dockerfile pins the CUDA base by digest and the Python package versions used by the release.
The static package was rerun through the complete 164-problem HumanEval+ Mini generation set and the frozen executable coding microbenchmark. Generated programs were executed in network-disabled containers. The model produced:
The 29/34 microbenchmark result exactly matches the earlier accepted K154 runtime. This is a bounded coding qualification, not a general benchmark claim.
Start with a small smoke request:
python3 bench/long_context_probe.py \
--base-url http://127.0.0.1:8000 \
--target-tokens 5000 \
--tokenizer-json /models/DeepSeek-V4.1-Flash-Next-DGX-Spark-512K/tokenizer.json \
--output smoke-5k.json
The exact accepted run takes about one hour on the tested machine:
python3 bench/long_context_probe.py \
--base-url http://127.0.0.1:8000 \
--target-tokens 522744 \
--max-output-tokens 128 \
--timeout 7200 \
--tokenizer-json /models/DeepSeek-V4.1-Flash-Next-DGX-Spark-512K/tokenizer.json \
--expected-prompt-tokens 524293 \
--prompt-token-tolerance 0 \
--expected-prompt-sha256 af9b0c94b91caf0fd53f6bc324236d3d94d9070740b88b66e7f25cf2c74e2c54 \
--expected-completion-tokens 44 \
--min-prefill-tok-s 145 \
--min-decode-tok-s 4 \
--output reproduction-512k.json
The probe generates deterministic varied archive records and inserts five unique values at roughly 5%, 25%, 50%, 75%, and 95%. It is a retrieval and operational-stability test, not a comprehensive long-context reasoning test.
| Component | Pinned identity |
|---|---|
| Source checkpoint | deepseek-ai/DeepSeek-V4.1-Flash |
| Source revision | dba1be0a40aa45a94ad051997016db3960a90277 |
| Upstream runtime | 0xBakeer/deepseek-v41-flash-spark |
| Upstream commit | 8b68fdde188fea5e8e9cd030f0e1b299dc16ac85 |
| K154 selection SHA-256 | 783ae482c5ffd5b8bdd154ec96462650e00452ca36e8f0cb74101c9ab703d811 |
| Release index SHA-256 | 09e45cbf8c2c370d4f7549af591c5c7375c9699abdfad49c8ea5bc615ed16561 |
| Exact 512K prompt SHA-256 | af9b0c94b91caf0fd53f6bc324236d3d94d9070740b88b66e7f25cf2c74e2c54 |
0xBakeer/deepseek-v41-flash-spark
and CREDITS.md.The runtime code is MIT licensed. DeepSeek-derived weights and reference files
retain their upstream license. See NOTICE before redistribution.
model-00048-of-00048.safetensors 102 GBmodel-00047-of-00048.safetensors 102 GBmodel-00046-of-00048.safetensors 2.7 GBmodel-00044-of-00048.safetensors 2.7 GBmodel-00045-of-00048.safetensors 2.6 GBk154-cb3/layers/layer-00.safetensors 2.2 GBk154-cb3/layers/layer-01.safetensors 2.2 GBk154-cb3/layers/layer-02.safetensors 2.2 GBk154-cb3/layers/layer-03.safetensors 2.2 GBk154-cb3/layers/layer-04.safetensors 2.2 GBk154-cb3/layers/layer-05.safetensors 2.2 GBk154-cb3/layers/layer-06.safetensors 2.2 GBk154-cb3/layers/layer-07.safetensors 2.2 GBk154-cb3/layers/layer-08.safetensors 2.2 GBk154-cb3/layers/layer-09.safetensors 2.2 GBk154-cb3/layers/layer-10.safetensors 2.2 GBk154-cb3/layers/layer-11.safetensors 2.2 GBk154-cb3/layers/layer-12.safetensors 2.2 GBk154-cb3/layers/layer-13.safetensors 2.2 GBk154-cb3/layers/layer-14.safetensors 2.2 GBk154-cb3/layers/layer-15.safetensors 2.2 GBk154-cb3/layers/layer-16.safetensors 2.2 GBk154-cb3/layers/layer-17.safetensors 2.2 GBk154-cb3/layers/layer-18.safetensors 2.2 GBk154-cb3/layers/layer-19.safetensors 2.2 GBk154-cb3/layers/layer-20.safetensors 2.2 GBk154-cb3/layers/layer-21.safetensors 2.2 GBk154-cb3/layers/layer-22.safetensors 2.2 GBk154-cb3/layers/layer-23.safetensors 2.2 GBk154-cb3/layers/layer-24.safetensors 2.2 GBk154-cb3/layers/layer-25.safetensors 2.2 GBk154-cb3/layers/layer-26.safetensors 2.2 GBk154-cb3/layers/layer-27.safetensors 2.2 GBk154-cb3/layers/layer-28.safetensors 2.2 GBk154-cb3/layers/layer-29.safetensors 2.2 GBk154-cb3/layers/layer-30.safetensors 2.2 GBk154-cb3/layers/layer-31.safetensors 2.2 GBk154-cb3/layers/layer-32.safetensors 2.2 GBk154-cb3/layers/layer-33.safetensors 2.2 GBk154-cb3/layers/layer-34.safetensors 2.2 GBk154-cb3/layers/layer-35.safetensors 2.2 GBk154-cb3/layers/layer-36.safetensors 2.2 GBk154-cb3/layers/layer-37.safetensors 2.2 GBk154-cb3/layers/layer-38.safetensors 2.2 GBk154-cb3/layers/layer-39.safetensors 2.2 GBmodel-00002-of-00048.safetensors 1.3 GBmodel-00043-of-00048.safetensors 1.3 GBmodel-00001-of-00048.safetensors 971 MBmodel-00017-of-00048.safetensors 186 MBmodel-00005-of-00048.safetensors 186 MBmodel-00011-of-00048.safetensors 186 MBmodel-00023-of-00048.safetensors 181 MBmodel-00027-of-00048.safetensors 176 MBmodel-00031-of-00048.safetensors 176 MBmodel-00035-of-00048.safetensors 176 MBmodel-00039-of-00048.safetensors 176 MBmodel-00013-of-00048.safetensors 170 MBmodel-00014-of-00048.safetensors 170 MBmodel-00015-of-00048.safetensors 170 MBmodel-00016-of-00048.safetensors 170 MBmodel-00018-of-00048.safetensors 170 MBmodel-00019-of-00048.safetensors 170 MBmodel-00020-of-00048.safetensors 170 MBmodel-00021-of-00048.safetensors 170 MBmodel-00022-of-00048.safetensors 170 MBmodel-00024-of-00048.safetensors 170 MBmodel-00025-of-00048.safetensors 170 MBmodel-00026-of-00048.safetensors 170 MBmodel-00028-of-00048.safetensors 170 MBmodel-00029-of-00048.safetensors 170 MBmodel-00030-of-00048.safetensors 170 MBmodel-00032-of-00048.safetensors 170 MBmodel-00033-of-00048.safetensors 170 MBmodel-00034-of-00048.safetensors 170 MBmodel-00036-of-00048.safetensors 170 MBmodel-00037-of-00048.safetensors 170 MBmodel-00038-of-00048.safetensors 170 MBmodel-00040-of-00048.safetensors 170 MBmodel-00041-of-00048.safetensors 170 MBmodel-00042-of-00048.safetensors 170 MBmodel-00003-of-00048.safetensors 170 MBmodel-00004-of-00048.safetensors 170 MBmodel-00006-of-00048.safetensors 170 MBmodel-00007-of-00048.safetensors 170 MBmodel-00008-of-00048.safetensors 170 MBmodel-00009-of-00048.safetensors 170 MBmodel-00010-of-00048.safetensors 170 MBmodel-00012-of-00048.safetensors 170 MB有文件?提交种子。
提交种子