DeepSeek-V4.1-Flash-1017-abliterated GGUF

The 763B uncensored DeepSeek-V4.1-Flash-1017, on one box.

236 GB in memory · 197B-parameter Engram streamed from your SSD · llama.cpp

OrcaRouter AI Gateway · X · Discord · GitHub · All Models

---

The full DeepSeek-V4.1-Flash-1017, run locally.

These are GGUF builds of the abliterated (refusal-removed) DeepSeek-V4.1-Flash-1017. The model is a 763B-parameter MoE with 384 routed experts (top-6), a 197B-parameter Engram n-gram memory, compressed sparse attention and a 1M-token context. The routed experts are the only part the bits were spent on: every layer's experts got their own precision from measured KL divergence, not a hand-set rule.

The Engram table is a lookup that reads 48 rows per token. llama.cpp maps it from disk and reads it on demand, so it lives on your SSD, not in RAM. What has to fit in memory is everything else.

---

⚠️ Disclaimer — read before use

This model is built on an abliterated base whose safety alignment has been substantially reduced: the refusal direction is orthogonalized out of every matrix that writes the residual stream. It will comply with many harmful, unethical, offensive, or illegal requests that the original DeepSeek-V4.1-Flash-1017 would refuse.

• It is released strictly for legitimate research: interpretability, AI-safety and refusal-mechanism study, red-teaming, and robustness evaluation.
• You assume full responsibility for how you use it and for everything it generates. Add your own safety, moderation, and abuse-prevention layers before any deployment.
• Use must comply with the MIT License inherited from the base model and with all applicable law.
• The authors accept no liability for misuse. Its outputs do not reflect the views of the uploaders or of DeepSeek.

---

Pick a build

Mix-256G is the build to use. It needs a box with 256 GB of RAM, unified memory, or VRAM. At 3.36 bits per expert weight, the model's next-token choice matches the unquantized model on 93.7% of held-out tokens.

Mix-128G fits 128 GB, but it is not a substitute. All experts at llama.cpp's smallest type (IQ1_S) already take 111 GB, so a 128 GB box is at the floor: perplexity is 55% higher and one token in five differs from the original. Use it to try the model, not to rely on it.

In memory is everything except the Engram table. Download adds the 110.6 GB Engram table, which stays on disk.

---

Evaluation

Both builds were scored against the lossless GGUF of the same checkpoint (MXFP4 experts, BF16 dense, MXFP8 Engram, converted bit-exact) on held-out text: wikitext-2 test, characters [0, 300000), 32 windows of 2048 tokens, full-vocabulary KLD. The precision search never saw this text. Each build is compared with llama.cpp's own mixture at a similar size, quantized from the same weights with the same importance matrix:

Measured with the Engram table at MXFP8. The released files carry it at IQ4_NL (110.6 GB instead of 202.8 GB). The swap was not rescored. Per unsloth's measurements on this model, the Engram precision barely moves top-1.

Method note. These are full-vocabulary KLD and next-token agreement on held-out text at 2k context, not a downstream benchmark.

Refusal and capability of the abliterated model

Measured on the unquantized abliterated checkpoint (orcarouter/DeepSeek-V4.1-Flash-1017-abliterated), not re-measured on these GGUF builds. They are copied here from that model's card. The KLD above is how far each build sits from that checkpoint.

Setup.

• Hardware and runtime: 8×H200, the checkpoint's own reference implementation under tensor parallelism.
• Decoding: greedy, thinking_mode="chat", 64 new tokens.
• Scoring: refusal is judged by a rule-based opening-phrase classifier. caveat means the model answered but wrapped the answer in a disclaimer. Treat the numbers as indicative, not publication-grade.

Safety: harmful-prompt refusal (lower = more uncensored). Weighted mean over all 500 prompts: 0.074, against 0.874 for the base checkpoint.

Over-refusal: benign prompts wrongly refused (lower = better). Weighted mean over all 350 prompts: 0.006, against 0.069 for the base checkpoint.

Capability retention.

MMLU is unchanged; the one-point difference is within the sampling error of 300 questions. It is the only capability measured.

---

How the precision was chosen

Each tier was searched separately, with the budget set on the bytes that have to be in memory:

1. Measure. For every group of tensors, the search measures the real KLD of moving it one precision level up and one level down. A group is one layer's experts, with gate+up and down kept apart, or one kind of dense matrix. Each measurement runs llama.cpp's own kernels.
2. Choose. A knapsack picks the combination of moves that fits the budget exactly.
3. Accept. Only steps whose candidate KLD was actually measured lower are kept. The search repeats until it converges.

The searched 256G build puts its bits where they count:

In the same budget, a uniform allocation scores KLD 0.146 and the searched one 0.045 (in-sample). Starting from the uniform allocation, upgrading any single one of the middle layers to MXFP4 removes ~0.04 KLD. Their errors compound downstream rather than add up.

In the 128G build there is almost nothing left to place. Experts are IQ1_S everywhere except a few later layers at IQ1_M or IQ2_XXS.

The other parts are set as follows:

• Attention and shared experts are Q5_K–Q8_0 in 256G and Q4_K–Q6_K in 128G.
• The router, norms and hyper-connection mixers are left as shipped.
• The importance matrix comes from 614k tokens of English/Chinese chat and Wikipedia.

---

What's in this repo

Both builds are text-only: the vision tower (mmproj) and the DSpark speculative drafter are not included. The DeepSeek-V4.1 chat template is embedded in each build.

---

Serving (llama.cpp)

DeepSeek-V4.1 is not in llama.cpp master yet. Build the PR that adds it, ggml-org/llama.cpp#30817. These files were made and tested at commit b72050d03:

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git fetch origin pull/30817/head:pr-30817 && git checkout b72050d03
cmake -B build -DGGML_CUDA=ON && cmake --build build -j --target llama-server llama-cli

hf download orcarouter/DeepSeek-V4.1-Flash-1017-abliterated-GGUF --include "Mix-256G/*" --local-dir ./dsv41-gguf

# GPUs with ~250 GB of VRAM in total (tested on 2 x H200)
llama-server -m dsv41-gguf/Mix-256G/DeepSeek-V4.1-Flash-1017-abliterated-Mix-256G-00001-of-00007.gguf \
  -ngl 99 -c 32768 -np 1 --no-reasoning-preserve

• Keep the files on an NVMe SSD and leave mmap on (the default). The Engram table is always kept off the GPU and read lazily. --mlock or --no-mmap would pull all 110 GB into RAM.
• 256 GB of system RAM and one GPU: add --cpu-moe to keep the routed experts in RAM and run attention on the GPU. This setup was not measured here.
• --no-reasoning-preserve drops earlier turns' thinking from the prompt, which is what DeepSeek's reference encoder does. With it, llama.cpp's rendered prompts match encoding.py token for token on DeepSeek's test cases.
• Thinking is on by default. Pass "chat_template_kwargs": {"thinking_mode": "chat"} for a direct answer, or "reasoning_effort": "low" | "medium" | "high" | "max" (or 1–100) to set its depth. The reasoning comes back in message.reasoning_content.
• Set a system prompt. Without one, the model sometimes misnames itself in thinking mode.

Measured on H200 (CUDA), short prompts, one request at a time:

CPU-only, Apple Metal and partial-offload speeds were not measured.

---

Model details

---

Bias, risks, and limitations

• Safety guardrails are substantially reduced (inherited from the abliterated base); see the disclaimer. The refusal and MMLU numbers above come from the unquantized checkpoint. Mix-128G in particular sits far enough from it (KLD 0.52) that its behaviour may differ.
• Mix-128G is heavily degraded. This is the limit of 1.6-bit post-training quantization on this model, not a tuning issue.
• The IQ4_NL Engram in the released files was not rescored. The table above was measured with MXFP8.
• Text-only. There is no vision projector and no DSpark drafter.
• Inherits the biases and limitations of DeepSeek-V4.1-Flash-1017. Long-context behaviour was not tested: all measurements are at 2k context or on short chats.
• Requires a llama.cpp build with PR #30817. Stock llama.cpp, Ollama and LM Studio will not load these files until it is merged.

License

MIT, inherited from the base model deepseek-ai/DeepSeek-V4.1-Flash-1017. The embedded chat template is unsloth's port of DeepSeek's encoder (unsloth/DeepSeek-V4.1-Flash-1017-GGUF, MIT).

---

One Gateway. Every Model.

Route Smarter · Ship Safer · Spend Less

Website · Models · GitHub · Discord · X