DeepSeek V4 Flash Abliterated

Ported to vLLM for massive throughput on RTX Pro 6000 — by Fraser Price

This is probs the best uncensored model you can run on 2 x RTX Pro 6000 at high throughput (source: ✨ my opinion ✨). Served on the voipmonitor vLLM fork it runs ~13× prefill and ~2.8× single-stream decode faster than the widely available abliterated GGUF versions.

---

Quick start (one command)

Requires Docker with the NVIDIA Container Toolkit, and ≥2 GPUs (see Hardware).

bash run.sh

run.sh defaults to this repo, so no arguments are needed. Everything is configured via env vars (see the table below) — e.g. GPUS=0,1,2,3 TP=4 bash run.sh to run on 4 GPUs.

run.sh (included) will:

1. Download the weights (~156 GB) into your Hugging Face cache (\~/.cache/huggingface, or $HF_HOME/$HF_HUB_CACHE) if not already present.
2. Pull the inference image if not already present.
3. Serve the model on http://localhost:8000/v1 (OpenAI API spec).

Optional — MTP speculative decoding (~1.8× single-stream decode speed). The checkpoint ships its MTP layer; enable it with MTP=2 bash run.sh (see the table below).

Override anything via env vars (all optional):

Example on 4 GPUs:

GPUS=0,1,2,3 TP=4 bash run.sh

---

Hardware/Image

Tuned for NVIDIA RTX Pro 6000 Blackwell (96 GB, sm_120). It will run on other Blackwell-class GPUs, but the numbers below and the default flags assume RTX Pro 6000. It needs roughly 170 GB of total VRAM for weights + KV cache at long context, split across GPUs by tensor parallelism.

The serving image targets Blackwell (sm_120) and CUDA 13.2. The DeepSeek-V4 architecture (sparse attention / lightning indexer, MLA, MTP, etc.) requires this image to run on RTX Pro. See local-inference-lab/rtx6kpro for image, bench, and serving details.

---

Performance

Measured under load on RTX Pro 6000 Blackwell (connected via PCIe 5.0), 128 generated tokens/request, TP = number of GPUs. MTP is speculative decoding (— = off, 2 = 2 draft tokens). PP / TG are prefill / per-request decode throughput (tok/s); Total is aggregate end-to-end throughput across all concurrent requests (tok/s). Rows are paired off/on so the MTP decode gain is directly comparable.

MTP=2 lifts single-stream decode by ~1.6–1.9× (e.g. 108 → 196 tok/s at 100k on TP=2) for a negligible TTFT cost.

---

How it was made

DeepSeek-V4-Flash's published abliterations existed only as a GGUF for the bespoke ds4/llama.cpp engines, which leave Blackwell hardware massively underutilised.

Rather than re-run abliteration from scratch, the refusal subspace was simply recovered and re-applied from an existing abliteration:

1. Recover the refusal directions. The rank-3 refusal subspace in the 4096-dim residual stream was extracted from the delta between huihui-ai's abliterated GGUF and the clean base.
2. Re-apply to the official weights. The same rank-3 projection was applied to every residual-writing matrix of the official checkpoint — attention output (attn.wo_b) and shared-expert down-projection (shared_experts.w2) across all 43 layers and the MTP layer (88 tensors total). Each tensor is dequantised → projected → requantised, with a fixed-point solver that compensates for FP8 requantisation so the intended projection survives quantisation.
3. Leave everything else untouched. The FP4-quantised routed experts and all gate/up projections are byte-identical to the official release; only the 88 FP8 residual-writing matrices change.

Validation: on every edited tensor the achieved projection matched target within ~0.002, and all 33,880 non-target tensors were verified byte-identical to the base. The served model drops refusals while keeping correct tool-calling and coherent step-by-step reasoning. No formal evals yet — tested manually; it should match the base huihui-ai abliteration at far higher speed.

---

Credits

• deepseek-ai/DeepSeek-V4-Flash — the base model (MIT).
• huihui-ai — the original abliteration (Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF), from which the refusal directions were derived (MIT).
• voipmonitor/vllm — the Blackwell/CUDA-13.2 vLLM build that runs the DeepSeek-V4 architecture on RTX Pro.
• local-inference-lab/rtx6kpro — the serving recipe (backends, flags, and benchmarks) this config is based on.

This checkpoint contains no huihui-ai weights — only the official DeepSeek-V4-Flash weights with a refusal-direction projection applied.

Disclaimer

This is an uncensored model: the refusal subspace has been removed, so it will not decline requests the way the base model does. It is released as-is, for research/lawful use only.

• Not a safety-aligned model. Because guardrails have been removed, outputs may be inaccurate, offensive, or harmful. Apply your own filtering, review, and safeguards before relying on or deploying it.
• No warranty. The weights and scripts are provided "as is", without warranty of any kind, express or implied. You use them entirely at your own risk.
• You are responsible for what you generate. The author does not endorse, encourage, or accept any responsibility or liability for how this model is used, including any unlawful, harmful, or unethical use. Outputs do not reflect the views of the author.

License

MIT, inheriting from the base model and the abliteration source.

---

Built by Fraser Price — @fraserpricee