DeepSeek V4.1 Flash on 3× DGX Spark

Uncensored, abliterated weights · 3,816 tok/s prefill · 124 tok/s code
Serving recipe on GitHub

3× DGX Spark · TP3 · 3.84 bpw EXL3 · about 274 GB · 1M context configured · GPUs capped at 2,300 MHz

Full specs: v1.2, measured 5 November 2026

Performance

Measured 5 November 2026 on production engine 30a9647 with sparkDash. All CPU cores in use, with performance cores capped at 3,600 MHz. Prefill used stock GPU clocks; decode capped GPUs at 2,350 MHz.

Prefill

Cold prompts, zero cached tokens, one request at a time; tok/s comes from time to first token with the collector's 171.55 ms request overhead subtracted.

Decode

Combined tok/s across all requests at once:

What each request gets, mean tok/s per request:

Near-empty KV pool, short cached prompts, 400 output tokens per request, one run per point, timed at the engine. Greedy is temperature 0 with thinking off; thinking is temperature 0.7 with thinking on. All 354 decode streams completed. Every point and per-stream timing: v1.2 benchmarks.

Clocks

Production serves with GPUs capped at 2,300 MHz, and the recipe does the same by default. The 3,816 headline is the stock-GPU run; at this cap 32K prefill is 3,711 tok/s, code decode 122 tok/s and 16-request prose decode 260 tok/s.

Each row is one full suite on engine 30a9647, with performance cores capped at 3,300–3,600 MHz. Other caps: clocks.csv.

Run it on three DGX Sparks

The tested setup is exactly three DGX Spark GB10 systems (128 GB each), every pair cabled directly over RoCE. Each rank file is about 93 GB, so the model needs all three. Budget about 440 GB of free disk per Spark (weights, rank file, Engram rows and kernel kit) and another 256 GiB on the head for the spill tier. Build the engine first, then follow the GitHub recipe:

cp cluster.env.example cluster.env && chmod 600 cluster.env   # HF_REPO and HF_REVISION already pin this repo
./scripts/prep-sparks.sh status   # firmware, 64K kernel, memory saver; apply, reboot, repeat
./scripts/preflight.sh     # check all three Sparks; changes nothing
./scripts/prepare.sh       # download this repo on each Spark, build and verify its rank file
./scripts/start.sh         # start the three ranks and the OpenAI-compatible API
./scripts/smoke-test.sh    # one API request

prepare.sh downloads this repository to every Spark, builds that Spark's rank file from tp3/rank<N>.keys and checks it against the SHA-256 values in the release manifest. Hardware, network and memory prerequisites: DGX Spark prerequisites.

start.sh caps every Spark's GPU at 2,300 MHz, the cap our production engine serves with. Set GPU_CLOCK_CAP_MHZ in cluster.env to pick your own, or 0 for stock clocks; ./scripts/clocks.sh set <MHz> changes it on running Sparks.

The folder is standard EXL3, but it has only been tested with the TensorFold dsv41 engine on this three-node setup.

Prefer to let an agent do it? The GitHub README has a copy-paste setup prompt.

Call it

The API speaks OpenAI Chat Completions and Responses plus Anthropic Messages, with streaming, tool calls and per-request thinking control. It listens on the head's loopback; call it there or through an SSH tunnel to port 18002.

set -a; . ~/ds1017/api/credentials.env; set +a   # on the head: loads SPARK_API_KEY without printing it
curl -s http://127.0.0.1:18002/v1/chat/completions \
  -H "Authorization: Bearer $SPARK_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"deepseek-v4.1-flash","messages":[{"role":"user","content":"Hello"}],"reasoning_effort":"high"}'

The thinking evals below ran at effort 50 (low); IFEval and the chat refusal tests ran with thinking off. Full reference: serving API.

Release v1.2

The current production recipe runs engine 30a9647, native profile v1.2, prefill layer slices and the serving API with native Messages routes. It keeps the same weights as v1.1. Long chat histories use far less memory when building a prompt, and a returning chat keeps its saved state while it waits in line, so it no longer recomputes from zero. See the changelog.

Fidelity

KL is mean per-token KL(official ‖ this model) against the published official-model logits (brandonmusic/DeepSeek-V4.1-Flash-1017-Teacher-Logits): 65 cuts of 2,048 tokens, 133,120 positions, balanced across agentic, chat, code, SWE, math, science, legal, multilingual, long-context, safety and pretraining text.

• Lower KL than the unedited 3.5 bpw quant on 59 of 65 cuts (paired mean 0.0088 lower, 95% CI 0.0035 to 0.0131).
• The same mrfakename edit scores 0.0937 on a 3.5 bpw base and 0.0784 on this 3.84 bpw mix.
• p99 KL 1.463. Every arm: fidelity.csv.

Quality and refusals

The production engine scored 97.1% (720 of 755) on the standard set measured 5 November 2026. Arithmetic, reasoning and tracking each reached 100%; GSM8K scored 96.5%, MMLU 94.4% and QA 92.0%. The task evaluations below measure the same weights on their own suites.

DeepSeek reports 90.9% GPQA-Diamond for DeepSeek-V4.1-Flash at maximum effort (100); these runs use effort 50. Quality runs used full-vocabulary sampling (top_k 0) except GPQA-Diamond (top_k 20, the serving default). GPQA ran mostly on engine 7d5b4d8 (greedy output identical) and AIME 2025 on 17800d4; the rest on 14803a3.

Chat runs use thinking off and 768 output tokens; 50 of the 100 harmful chat replies reached that cap. Thinking runs use 3,072 tokens; 35 of the 50 harmful thinking replies hit that cap, which is where most partials come from. HarmBench's own classifier (cais/HarmBench-Llama-2-13b-cls) rates 97 of the 100 chat answers as complied. On all 250 XSTest safe prompts, a local Qwen2.5-7B-Instruct judge with the published rubric found 0 full refusals and 61 partial ones (24.4%); 55 of those are nonsense-premise prompts, where XSTest's rubric counts rejecting the premise as compliance. Data: quality.csv.

Files

How it was built

The pack starts from easthambuilds' 3.75 bpw EXL3 base and replaces selected tensors with LibertAIDAI, bertholomus and LakyFx quantizations. Attention, shared experts and selected gate/up components use higher precision; the least-used quarter of experts in layers 10–37 uses 3-bit components.

mrfakename's rank-2 refusal-removal edit changes attention wo_b in layers 0–31 and shared-expert w2 in layers 10–39. It was applied to the official FP8 weights in FP32, then the affected tensors were re-encoded at 6 bits with EXL3 mul1 K6 and an identity Hessian. The edit is baked in; there is no runtime switch.

Tensor ranges and the bitrate method: weight provenance.

Intended use and risks

This model has had its refusal behavior largely removed; the refusal tables above show what remains. It will answer requests the official model declines, including harmful ones, and it can be wrong, biased or unsafe. Use it for research into quantization fidelity, serving and refusal behavior, and for applications where you add your own safeguards. You are responsible for how you deploy it and for complying with the laws that apply to you. Keep the API behind access controls, keep credentials separate and have people review consequential outputs; see the security guidance.

Tested scope is text on exactly three DGX Sparks. The base model's image input has not been tested with these weights.

License and credits

MIT, following DeepSeek's base-model license. Every source above publishes under MIT. Provided as is, without warranty of any kind.

Thanks to DeepSeek; easthambuilds, LibertAIDAI, bertholomus and LakyFx for the EXL3 tensors; mrfakename for the edit; turboderp for EXL3; Ash Hart and BertholomusAI for TensorFold; Mia's AI Lab for sparkDash; and brandonmusic for the official-model logits. Full list: CREDITS.md.

Built by ChimeraWerks: weight composition, TP3 engine work, serving and every measurement of this model on this card.