DeepSeek-V4.1-Flash-1017-heretic-GGUF

GGUF builds of pqhaz/DeepSeek-V4.1-Flash-1017-heretic, our refusal ablation of deepseek-ai/DeepSeek-V4.1-Flash-1017. This is not an official DeepSeek release.

It is intended for authorized security research in environments the researcher owns or has permission to test.

Needs a llama.cpp fork

These files do not load in upstream llama.cpp. As of this release, upstream has no DeepSeek-V4.1 architecture. They were made and tested with vcruz305/llama.cpp:

• converted with branch feat/deepseek-v41-convert (2076412, upstream PR #28696);
• run with branch runtime/deepseek41 (5210c7c).

The files will need re-conversion if the format changes before the architecture is merged upstream.

Files

Both builds were regenerated with the same commits after the first machine was lost. Every tensor in the Q6_K splits that were uploaded earlier matches the new build byte for byte, so the set is consistent.

The released routed experts are already MXFP4, and both builds keep them bit-for-bit as released. Only the attention, shared experts, Engram tables and embeddings are K-quantized. This also means a smaller K-quant would barely shrink the file: the experts are most of it (Q8_0 is 508 GB, Q6_K 457 GB).

Only the language model is included, with no vision tower and no MTP/DSpark layers.

Download a whole folder and point the runtime at the first split, for example:

llama-server -m Q6_K/DeepSeek-V4.1-Flash-1017-heretic-Q6_K-00001-of-00011.gguf -ngl 999 -c 16384 \
  --jinja --chat-template-file chat_template.jinja

Pass --chat-template-file chat_template.jinja (in this repo). The template embedded in the GGUF is DeepSeek's V4.1 template, and llama.cpp's Jinja engine rejects it: it uses dict(), ** unpacking and chained comparisons. chat_template.jinja here is the same template rewritten without them. With Hugging Face's Jinja setup it renders byte-identical prompts to the original on 96 test conversations, covering tools, tool results out of order, namespaces, thinking and chat mode, and every reasoning_effort form. llama.cpp's engine renders the same 96 byte-identically.

Tool calls: with this template, llama-server's automatic parser handles V4.1's <｜DSML｜ calls> format and returns structured tool_calls (tested below).

Tested

The Q8_0 build was served with llama-server from the runtime branch on 4× B300 (CUDA, sm_103) with this chat template, at temperature 1.0 and top_p 0.95:

The Q6_K build was produced from it with llama-quantize --allow-requantize --tensor-type "ffn_(up|gate|down)_exps=mxfp4". It was loaded the same way on 3× B300 and passed the same three requests: the essay finished on its own after 2472 tokens with no loop (worst window 2.0×), and the tool call parsed. Its decode speed was 48–56 tok/s.

Q6_K compared with Q8_0, measured with llama-perplexity --kl-divergence at 32 × 2048 tokens:

Refusal and long-output measurements are in the main model card and were made on the original weights. They were not repeated on the GGUF files. No capability benchmarks were run.

Credits

DeepSeek for the model; vcruz305 for the DeepSeek-V4.1 conversion and runtime; Heretic for the method.