NIGHTFALL TP2

Before you download: BLACKFOLD is coming.

NIGHTFALL TP2 was put together quickly as an initial EXL3 build. I am working on BLACKFOLD, a TensorFold version for TP2, which I expect to be a substantially better release. My recommendation is to wait for BLACKFOLD if you are deciding which version to set up.

This build remains available if you want to try it now; the choice is yours. The benchmarks below apply to NIGHTFALL TP2. BLACKFOLD results will be published once it has been tested.

DeepSeek V4.1 Flash 1017 · EXL3 2.9 bpw · Two-node deployment

A refusal-reduced DeepSeek V4.1 Flash 1017 build for tensor-parallel inference across two DGX Sparks.

NIGHTFALL TP2 is a derivative of the 763B-parameter MoE model, using the EXL3 2.9 bpw base and targeted attention-output edits. This release distributes a 997 MB weight patch and reproducible assembly scripts. It re-encodes the attention-output edits from the pinned Heretic source into the existing EXL3 pack. It does not contain a standalone model or the complete base weights.

The assembled model was evaluated against the unmodified EXL3 base on two DGX Sparks. The observed reduction in refusals is substantial, with small changes on the measured perplexity samples. Broad capability benchmarks have not been run. Zero refusals on an individual test set does not establish universal non-refusal or correct answers.

Release at a glance

Modification scope

The patch replaces only the encoded representation of the 38 attention-output projections. Routed experts, shared experts, Engram tables, embeddings, output head, and non-target tensors retain their base values. The assembly script verifies the inserted tensors and compares every non-patch tensor in each rewritten shard. Re-serializing a shard changes its file bytes; the preservation claim applies to non-target tensor contents.

The edits originate from the pinned Heretic derivative. The patched FP8 matrices were dequantized and re-encoded with exllamav3 1.6.2, K=5, the pack's mul1 codebook, and out_scales=always. The quantizer used its uncalibrated q_fallback path because the original pack's calibration data was unavailable.

Mean re-encoding RMSE was 0.000855; mean round-trip normalized MSE was 0.001151. These measure weight reconstruction, not downstream model quality.

Layers 37–39 are included in the edit and feed speculative drafting. Separate drafter parameters are not targeted by the patch, but the drafting path cannot be described as wholly unaffected. Acceptance was broadly comparable in the measured phases; this is not a guarantee for other prompts or runtimes.

Evaluation

Measurements were recorded on 18 October 2026. Unless stated otherwise, the comparison is between the unmodified LibertAIDAI EXL3 pack and the same pack with this patch, using the same two-node hardware, harness, prompt sets, and sampling settings. It is not a comparison against the full-precision upstream model.

Refusal counts: stock versus NIGHTFALL TP2

Labels were assigned by a separate Qwen3.6-35B-A3B REAP-19B model. They are automated judgments, not independent human adjudication. A non-refusal may still be partial, incorrect, or truncated.

With thinking enabled, 9 of 200 NIGHTFALL TP2 responses produced no final answer before the 4,096-token limit. These were labeled NO_ANSWER, not refusals. Overall, 19/200 NIGHTFALL TP2 thinking-on responses reached the limit, versus 0/200 stock. On Bolt-169, 112/169 NIGHTFALL TP2 responses reached the generation limit, versus 43/169 stock. These limits affect interpretation of answer completeness.

HarmBench-320: NIGHTFALL TP2 only

Official text test split; direct requests, greedy generation, thinking off, maximum 512 new tokens. Standard/contextual outputs were evaluated with cais/HarmBench-Llama-2-13b-cls; the copyright subset used the official MinHash matching procedure. These attack-success rates are benchmark-specific and are not general accuracy scores. No matched stock run is reported for this 320-prompt evaluation.

StrongREJECT: NIGHTFALL TP2 only

313 prompts; greedy generation, thinking off, maximum 512 new tokens. 312 responses were parsed and one was unparsed. Refusal rate was 8.65% and mean rubric score was 0.8818 on a 0–1 scale.

The evaluation used the StrongREJECT rubric with mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit-REAP-19B, not the official fine-tuned StrongREJECT judge. The score is a rubric proxy, not a general measure of answer usefulness or a directly comparable official leaderboard result.

Perplexity, distribution drift, and repetition

KL was estimated from each model's top-20 log probabilities plus a residual tail bucket; it is not full-vocabulary exact KL. The repetition check used a compression-ratio heuristic with up to 12,000 generated tokens per prompt.

Hermes and the broader capability packs have not been run. These checks do not establish unchanged reasoning, coding accuracy, factuality, or long-context retrieval quality. Raw response transcripts and the complete private evaluation harness are not distributed in this repository, which limits independent reproducibility of the reported scores.

Performance and deployment limits

The validated setup used two DGX Sparks over a high-speed interconnect, TP2, DSpark speculative decoding at k=3, and up to two concurrent sequences. A 600,000-token context configuration was used; the reported prefill tests reached approximately 30,000 tokens. The configuration is not evidence of evaluated quality across a full 600k context.

Decode used temperature 1.0, top-p 0.95, and thinking off. These are runtime-optimization comparisons on the NIGHTFALL TP2 model, not speed gains caused by removing refusal directions. The optimized result used a custom cooperative-MoE binary that is not included in this repository; installing this patch alone does not reproduce that throughput.

For the matched stock-versus-NIGHTFALL TP2 run before those optimizations, mean prose decode was 27.01 versus 27.26 tok/s, respectively. Hardware, runtime patches, context, concurrency, and prompts all affect performance. See serving notes.

Installation

This is an advanced EXL3 deployment. It is not a Transformers from_pretrained checkpoint, a GGUF, or a standalone server package. Obtain access to this gated repository, then download it with the Hugging Face CLI:

hf auth login
hf download BlackwellBoy/DeepSeek-V4.1-Flash-1017-NIGHTFALL-EXL3-2.9bpw-TP2 --local-dir ./nightfall-tp2
cd nightfall-tp2
bash setup.sh /path/to/models

Prerequisites: Linux/Bash, Python 3 with PyTorch and safetensors, the hf CLI, sha256sum, and sufficient local NVMe capacity. The setup downloads the pinned bases, verifies inputs, applies the patch in place to the downloaded EXL3 directory, and checks the resulting shard hashes. It does not install or launch an inference engine. Use a dedicated destination; do not point it at a live model directory.

Output layout:

/path/to/models/
  exl3-2.9bpw-heretic/   # assembled 43-shard EXL3 checkpoint
  engram-src/           # source Engram tables and embed-only index

Disk footprint per node

Allow at least 420 GB plus operational headroom for assembly without packed Engram, and more when preserving separate stock/output copies. Rewriting affected shards into a separate directory requires roughly 190.5 GB extra. The measured TP2 setup kept a full local copy on each node. Packed Engram supplements the source tables; it does not replace them in the tested runtime. Disk size is not a VRAM requirement.

Pinned sources and integrity

Weight-patch SHA-256:

f42912922985684e7a0cd8f3f6d73efafec620399e8100f09cfdb0ab67b0e649

stock_shards.sha256 records the 43 input shards; SHA256SUMS records the patch and assembled output shards. GRAFT_META.json records the pinned revisions and edit configuration.

Attribution and license

This independent release contributes the EXL3 re-encoding, assembly scripts, integrity manifests, and deployment measurements. The base model is from DeepSeek-AI, the EXL3 base quantization from LibertAIDAI, and the refusal-direction edits from pqhaz. The method uses Heretic; the format and quantizer are from exllamav3. This is not an official release from those projects.

The patch, scripts, and documentation are distributed under MIT. Upstream components retain their respective licenses. Heretic source is AGPL-3.0 and is not redistributed here.

Download access remains manually reviewed under the repository's Responsible Use Agreement. Access conditions and software licensing are described separately. The model is provided without warranty; deployers are responsible for evaluating suitability and applying the controls their use case requires.