WARMIND-1017-FLARE-gb202

WARMIND-1017, prepared as a Flare runtime for four RTX PRO 6000 Blackwell GPUs.

WARMIND-1017 is jcartu's abliterated release of DeepSeek-V4.1-Flash-1017. The refusal direction is inverted in 132 dense tensors and nothing else is touched.

This repository is that model exported to Flare. Flare is the native Rust serving runtime from NVIDIA's spark-local-inference project, led by Luke Alonso of Local Inference Lab. The package uses the same gb202-tp4 layout as Local Inference Lab's own DeepSeek-V4.1-Flash-1017-FLARE gb202 package, and it targets the Flare image Local Inference Lab publishes.

This revision is built on Flare 78b18b1c and FlashInfer e977747a, the same commits as Local Inference Lab's current gb202, with the clustered PCIe all-reduce ladder. Download it and run it; there is no hour-plus export to do yourself. The previous build, for the older release-6fa01a4-main-20261101 image, stays available on the gb202-6fa01a4 branch.

At a glance

How it differs from LiL's gb202 package

LiL's current gb202 (revision 32385269 of November 4, 19:08 UTC) is built on the same Flare and FlashInfer commits, with native expert scales and runtime-tuned clustered collectives, like this one.

• Weights. WARMIND-1017 instead of stock DeepSeek-V4.1-Flash-1017. The files for the untouched layers and the Engram tables come out byte-identical to LiL's (see License). The edited layers 4 to 36 differ.
• Export settings. joninco's settings: NVFP4 draft head, FP8 Markov head and vocabulary-parallel sampling, with BF16 MLA compute.
• KV capacity. On our box this package holds 16,807,936 KV tokens. LiL's card for its current gb202 gives about 17.0 million on four 96 GB GPUs; we have not measured that revision here.
• Sampling. Like every DeepSeek V4.1 Flare package, temperature, top-k, min-p and the presence and frequency penalties are fixed at their defaults. A request that sets them is served at the default, not rejected.
  • top-p is fixed at 1.0 as well. Vocabulary-parallel sampling does not support a lower top-p, and PR #180 makes Flare serve a lower top-p at 1.0 instead of failing the request. LiL's current gb202 is built without PR #180 and declares Flare's DeepSeek default top-p range of 0.95 to 1.0.
  • Repetition penalty can be set from 1.0 to 1.1, as in LiL's current gb202.
• Signing. Signed with our own development key, not Local Inference Lab's. Flare asks you to trust it once (see below).

What changed from the previous build

The previous build (gb202-tp4-202611030828, now on the gb202-6fa01a4 branch) was exported at Flare cc696a6b with statically tuned collectives, for the release-6fa01a4-main-20261101 image. This one was exported at Flare 78b18b1c with FlashInfer e977747a and the clustered PCIe all-reduce ladder from #174, tuned when the package starts.

• Weights. Unchanged. 170 of the 185 package files, 521 GB, are byte-identical to the previous build, including every weight file. Only the per-rank deployment files (kernels, graphs and initial state), the manifest and export.json are new.

• Speed. Both builds' release checks ran on their downloaded copies with the same command, except for the image, on different nights:

  	Previous build	This build
  Festr's single-user test	493 tok/s	544 tok/s
  llm_decode_bench C1	319	299
  C8	1,013	1,144
  C32	1,806	2,071
  KV capacity	17,519,872	16,807,936

  Concurrent throughput is clearly higher. In every comparison of a clustered build with a static one, the C1 cell of llm_decode_bench came out 2–8% lower for the clustered build, while C8–C32 came out higher. Festr's single-user test moved both ways: higher than the previous release here, lower than the 18b16e17 build in the pre-upload A/B below. We report all of it.

• KV capacity. Runtime collective tuning used to keep about 3.2 GiB per GPU of startup-only storage. Our intermediate build on Flare 18b16e17 held only 13,551,360 KV tokens for that reason. #197 and FlashInfer #72 share that storage, so this build holds 16.8 million, about 4% below the static previous build.

• Image. Needs release-18b16e1-main-20261104 or newer (LiL: older images cannot load packages from this Flare).

• CSF sibling. WARMIND-1017-FLARE-CSF-gb202 was rebuilt the same way and holds 20.8 million KV tokens.

• Not in this build. Flare 5a1640c8 (November 5) moves to FlashInfer 1f8ed8a3, which clears unused KV operands in the sparse-MLA attention kernels before the arithmetic. Its commit notes that the SM120 paths were not tested. LiL's gb202 at the time of writing does not include it either. We will rebuild when LiL does.

Download and run

Download the package with the Hugging Face hf CLI, check it, then serve it with the Flare image:

mkdir -p models flare-state
hf download jcartu/WARMIND-1017-FLARE-gb202 --revision gb202 --local-dir ./models
(cd models && sha256sum -c SHA256SUMS)
echo "flare:x:$(id -u):$(id -g)::/state:/sbin/nologin" > flare-state/passwd

docker run --rm -it --user "$(id -u):$(id -g)" -w /state \
  -v "$PWD/flare-state/passwd:/etc/passwd:ro" \
  --gpus '"device=0,1,2,3"' -e CUDA_DEVICE_ORDER=PCI_BUS_ID \
  --ipc=host --ulimit memlock=-1:-1 --ulimit nofile=1048576:1048576 \
  --security-opt seccomp=unconfined -p 127.0.0.1:8000:8000 \
  -v "$PWD/models:/models:ro" -v "$PWD/flare-state:/state" \
  ghcr.io/local-inference-lab/flare:release-18b16e1-main-20261104 \
  run /models/model.safetensors \
  --publisher-keys /models/development-publisher.json \
  --trust-store /state/trust.toml --gpus 0,1,2,3 \
  --max-concurrency 32 --mtp-draft-tokens 7 --mtp-adaptive-verification \
  --mtp-adaptive-verification-alpha 1 --scheduler-policy throughput \
  --embedding-table-storage disk --embedding-table-cache-size 32GiB \
  --embedding-table-resident-scales --kv-cache-limit memory \
  --host 0.0.0.0 --port 8000

• Download. On our connection the full download took 15 minutes. The checksum step reads all 524 GB again; skip it if you trust the download.
• Older image. For release-6fa01a4-main-20261101, download with --revision gb202-6fa01a4 and use that image in the command; that branch's card has its numbers.
• Trust prompt. On first start Flare shows this package's signing key and asks you to type local-development.
  • What it trusts: this one key, not development keys in general. The choice is remembered in flare-state/trust.toml.
  • License line: the prompt shows License: mit, which refers to the DeepSeek weights.
  • Unattended starts: add --dangerously-approve-key fa77427544ec0959c79734f8fb5632e2c1ac97c674ada5e07d6ba127b550ce68 --dangerously-accept-publisher local-development, which accepts exactly this key.
• The passwd line and -w /state. The release image only knows its own flare user. When Docker runs as your user ID, Flare cannot find a home directory and stops with cannot resolve home directory, or with Permission denied from the image's working folder. These two lines give your user a home in flare-state. Local Inference Lab's own package hits the same error on this image when run that way.
• --publisher-keys points Flare at this package's public key. Without it Flare stops with package signer is not a configured publisher.
• Other flags.
  • --ipc=host and seccomp=unconfined are what we ran with. The latter lets Flare use io_uring for disk-backed Engram inside Docker.
  • --embedding-table-resident-scales and --kv-cache-limit memory are Flare's defaults, spelled out.
• Engram in RAM. With about 256 GiB of host RAM, replace the three --embedding-table-* flags with --embedding-table-storage ram.
• Model name. /v1/models reports the export folder name, DeepSeek-V4.1-Flash-1017-abliterated-native-overnight, as the model's root. Requests use DeepSeek-V4.1-Flash.
• Downloading through Flare. Flare can fetch from the Hub itself (run jcartu/WARMIND-1017-FLARE-gb202 --revision gb202 --filename model.safetensors), but we did not take that path to the end. Use hf download.
  • Speed: in a 3-minute trial it ran at about 15 MB/s here.
  • Key: it would still need --publisher-keys pointing at the downloaded development-publisher.json.
  • Warning: it will warn that the package names local-development rather than this repository as its source.

Send a request

The OpenAI-compatible API serves the model as DeepSeek-V4.1-Flash:

curl --fail http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"DeepSeek-V4.1-Flash","messages":[{"role":"user","content":"Explain how a hash table works."}],"max_tokens":1024}'

• Sampling. Defaults are temperature 1.0, top-p 1.0, top-k off, min-p 0, repetition penalty 1.0, and presence and frequency penalties 0. Only repetition_penalty (1.0 to 1.1) and seed are request-controlled; any other sampling value you send is clamped to its default.
• Thinking. Turn it on or off with "chat_template_kwargs": {"enable_thinking": true}.
• Images. The export includes the vision path (--image-concurrency 4, four image-bearing requests at a time, like gb202), and /v1/models lists image input. We did not test image requests on this package.

Tips from running it

• Put the Engram files on XFS or ext4, not btrfs. btrfs routes these small direct reads through extra kernel threads. With the same package copied to both filesystems and the same flags (measured on our sibling export of these weights), a fresh 106k-token prompt took about 10.5 s from btrfs RAID0 (two PCIe 5.0 drives) and about 8 s from XFS on a single PCIe 4.0 drive.
• A bigger row cache helps text the model has seen before. We run --embedding-table-cache-size 64GiB in production. It does not speed up the first read of brand-new text.
• Adaptive-verification alpha 1. LiL's card uses --mtp-adaptive-verification-alpha 0.6. On our box (no PCIe switch, each GPU on its own host bridge), 0.6 was about 5% slower than 1 at 16 concurrent requests (1,425 vs 1,510 tok/s, measured with our November 2 export of these weights on the same image), so the command above uses 1. Try both on switched systems.
• Throughput scheduler for concurrent work. It is the right choice for agent swarms and parallel requests. Keep --max-concurrency 32 unless you need memory back.

Measured

Every number below was measured on the files downloaded from this repository and served with the command above (unattended trust flags, a different host port), on release-18b16e1-main-20261104 (downloaded from the staging branch that became gb202). Rig: 4x RTX PRO 6000 Blackwell Workstation Edition, Xeon w9-3495X.

The quality and refusal evaluations of the WARMIND edit itself are in EVALUATION-1017.md: MMLU within noise of stock, a KL diagnostic, and blind-labelled refusals. They were run on the vLLM CSF container, jcartu/DeepSeek-V4.1-Flash-1017-abliterated-lossless-CSF, whose 132 edited tensors are byte-identical to the ones exported here.

How it was built

1. Source. The 82 weight and runtime files of jcartu/WARMIND-1017 were hashed and matched against that repository's SHA256SUMS.

2. Export. Flare's Ignition exported the package at Flare commit 78b18b1c (November 4), the commit LiL's current gb202 was prepared from. Packages from this commit load on the release-18b16e1-main-20261104 image.

   • #180 (the top-p fix above) was applied locally, from the pull request's current head 38963a4b.
   • #154 by Brandon M. Music, which lets a 32-request export fit in 96 GB of VRAM, is already merged at this commit.
   • FlashInfer e977747a is FlashInfer main with #71 and #72 merged, the revision LiL's current gb202 was prepared with. Flare's own pin at this commit, ddbe93fc, is the second of #72's three commits.

   The full command, commits and settings are in export.json. Autotuning ran at Ignition's default effort.

   ignition export <WARMIND-1017> --gpus 0,1,2,3 \
     --max-concurrency 32 --prefill-chunk-size 4096 --image-concurrency 4 \
     --mtp-draft-tokens 7 --speculative-token-capacity 128 --speculative-graph-budget 64 \
     --mla-compute-mode bf16 --draft-head-quantization nvfp4 --markov-head-quantization fp8 \
     --vocabulary-parallel-sampling --expert-scale-compression native \
     --collective-tuning runtime \
     --collective-ladder ignition/preparation/ladders/gb202_pcie_tp4_clustered_full.json \
     --compile-workers 8
   

3. Pre-upload checks. Before upload, the package was served on the release image back to back with our then-production build (WARMIND on Flare 18b16e17, clustered collectives), with the same flags. This build held 16.76M KV tokens against 13.55M. It ran at 303, 1,087, 1,589 and 2,018 tok/s at C1, C8, C16 and C32, against 323, 1,160, 1,656 and 2,084, and at 527 against 542 tok/s on Festr's test. In that A/B the production package was read from XFS and this one from btrfs.

4. Release checks. After upload, it was downloaded back from this repository with the command above, and every package file was matched against SHA256SUMS. The Measured numbers were taken on that download.

Files

Keep the files together under their original names.

Lineage and credits

• Base weights: DeepSeek-AI, deepseek-ai/DeepSeek-V4.1-Flash-1017 @ 1bae0e1088993b9b82de08f59de5be5a34e42170, MIT.
• Method: grimjim, Norm-Preserving Biprojected Abliteration.
• Refusal directions and the original s3.0 recipe: distributedcognition/DeepSeek-V4.1-Flash-abliterated (norm-preserving biprojection, scale 3.0, layers 4 to 36), built on the previous DeepSeek-V4.1-Flash.
• WARMIND-1017: operator re-implementation, validation, 1017 re-abliteration and evaluation by jcartu (WARMIND-1017).
• Flare: the runtime and the export toolchain, by NVIDIA and the Flare community contributors, led by Luke Alonso of Local Inference Lab.
• Local Inference Lab: the gb202 runtime layout and the release image this package targets, from Luke Alonso and Festr. Their official DeepSeek-V4.1-Flash-1017-FLARE package is the reference this one follows.
• Martin Vit (voipmonitor): the clustered PCIe all-reduce collectives and ladders this build is tuned with (Flare #174, FlashInfer #70) and the shared startup tuning storage (Flare #197, FlashInfer #72) that gives it back its KV capacity.
• Mark Yong: the fused-MoE fix in FlashInfer #71, part of FlashInfer e977747a.
• joninco: the export settings that bring a self-exported package to gb202 speed.
• Brandon M. Music: the export memory fix in #154.
• FlashInfer and the b12x authors: the kernels Flare compiles into this package (FlashInfer e977747a).
• This package: export, qualification, the top-p fix and these documents by jcartu on a 4x RTX PRO 6000 build machine. Packaging review by Claude Fable.

License

• Weights. The DeepSeek-V4.1-Flash-1017 weights are under the MIT License (LICENSE, Copyright (c) 2023 DeepSeek). The WARMIND edit adds no terms.
• Compiled runtime components. The package's compiled CUDA kernels and captured graphs were produced by Flare, under the Apache License 2.0 (LICENSES/Flare-Apache-2.0.txt, Copyright NVIDIA CORPORATION & AFFILIATES and Flare community contributors).
  • FlashInfer/b12x kernels are included, also Apache 2.0 (LICENSES/FlashInfer-Apache-2.0.txt).
  • CUTLASS and FlashAttention-3 components bundled with FlashInfer are under the BSD-3-Clause license (LICENSES/CUTLASS-BSD-3-Clause.txt, LICENSES/FlashAttention3-BSD-3-Clause.txt).
  • See NOTICE for the full attribution.
• Local Inference Lab. No Local Inference Lab license applies here. This is not a Local Inference Lab release, and nothing from LiL's CSF containers is in it.
  • What is shared: 36 of the package's weight files are byte-identical to files in LiL's MIT-licensed gb202 package (revisions cefb84d2, e659bf6c, b3c7e7c6 and 32385269): the four Engram files and, for each rank, the weight files for the untouched layers 0–3 and 37–39 plus misc, 274.6 GB in all. (LICENSE is DeepSeek's MIT text in both repositories.)
  • Why: Ignition serializes the same DeepSeek bytes the same way, and its content-addressed cache reused files it already held after this export had verified its own source bytes.
  • What is new: the 132 rank files for layers 4–36 are the WARMIND edit (unchanged from this repository's previous build). The kernels, the graphs and the manifest were produced by this export.
  • Check it yourself: compare the weight files' SHA-256s on the Hub. The weight files that differ are exactly the WARMIND edit; the kernels, graphs and manifest differ because they are this export's.