DeepSeek-V4.1-Flash-1017-heretic-NVFP4
pqhaz/DeepSeek-V4.1-Flash-1017-heretic, our refusal ablation of deepseek-ai/DeepSeek-V4.1-Flash-1017, with the routed experts in NVFP4. It uses the layout of nvidia/DeepSeek-V4.1-Flash-NVFP4 at revision 3431dde (FP8 Engram tables, the revision NVIDIA validated in SGLang). This is not an official DeepSeek or NVIDIA release.
It is intended for authorized security research in environments the researcher owns or has permission to test.
How it was made
No weight was re-quantized. The released routed experts are MXFP4: E2M1 values with one power-of-two (E8M0) scale per 32 values. This build keeps the same E2M1 values and writes each scale 2^k as an E4M3 scale per 16 values times one FP32 scale per matrix, 2^m. Here m is the largest k of the matrix minus 8, with w1 and w3 of an expert sharing one m. Negative zero is written as +0.
This is exactly how NVIDIA's export encodes the original V4.1-Flash weights. Run on the original V4.1-Flash, our converter reproduces NVIDIA's bytes for all 384 experts checked: 0 of 3,456 tensors differ. The script is nvfp4_repack.py in this repository.
- Activation scales (
input_scale, needed for W4A4) cannot be derived from the weights. They are taken from NVIDIA's checkpoint, which was calibrated on V4.1-Flash, not on 1017. - Everything else is byte-identical to
pqhaz/DeepSeek-V4.1-Flash-1017-heretic: attention (including the ablatedwo_b), shared experts, Engram tables, vision tower, MTP/DSpark layers and tokenizer.
Serving
SGLang, the image NVIDIA tested (lmsysorg/sglang:dev-cu13-dsv41), on 4 Blackwell GPUs:
python3 -m sglang.launch_server --model-path <dir> --tp 4 --context-length 262144 \
--reasoning-parser deepseek-v41 --tool-call-parser deepseekv41 --chunked-prefill-size 4096 --trust-remote-code
Thinking is off by default in this build. Enable it per request with "chat_template_kwargs": {"thinking": true} or "reasoning_effort": "max".
Measured
On 4× B300 at temperature 1.0 and top_p 0.95, with the same tests as the full-precision build:
Full-precision build (SGLang dev-dsv41, TP4 + EP4) |
This NVFP4 build (SGLang dev-cu13-dsv41, TP4) |
|
|---|---|---|
| Refusals with reasoning on, JBB-Behaviors held out (100 prompts, 6000-token limit, LLM judge on the final answer) | 1 | 1 |
| Median completion tokens on those prompts | 1517 | 1461 |
| Long ordinary outputs (8 tasks × 16, 12,000-token limit): loops | 0 / 128 | 0 / 128 |
| Long ordinary outputs: flagged by any check | 0 / 128 | 1 / 128 |
| Long ordinary outputs: reached the limit | 48 | 45 |
| Single-stream decode, 1500-token essay | ~220 tok/s | ~215 tok/s |
The one flagged NVFP4 output is a code-writing response that is 7% whitespace, just under the 8% threshold for fused words. In the same task the full-precision build's outputs go down to 8–9%, and the response has no loop: its most repetitive 2000-character window compresses 3.1×.
The base model refuses 92 of the same 100 prompts in the same setup. No capability benchmarks were run.
Credits
DeepSeek for the model; NVIDIA for the NVFP4 layout and activation scales; Heretic for the method.