DeepSeek-V4-Flash-Vision-Exp-Abliterated-GGUF
GGUF quantizations of
windowsxp811203/DeepSeek-V4-Flash-Vision-Exp-Abliterated
— the refusal-ablated build of deepseek-ai/DeepSeek-V4-Flash-Vision-Exp. See the parent repo for the
abliteration recipe and the full refusal / capability evaluation. Refusal on harmful requests is 0.00%
(0/520 AdvBench, 0/300 HarmBench harmful categories, hand-adjudicated).
These are text-only. llama.cpp's DeepSeek-V4 support does not include the vision tower, so the converter drops it; use the safetensors parent with a vision-capable runtime if you need image input.
Files
| Quant | Size | Notes |
|---|---|---|
MXFP4 |
156.4 GB (4 shards) | The routed experts are already MXFP4 in the FP8 source, so this is the closest thing to "native" and is smaller than a Q4_K_M would be. |
Q3_K_M |
135.3 GB (4 shards) | Smallest level worth shipping. |
Both were verified to load and generate on a CUDA build of llama.cpp (≈58 tok/s MXFP4, ≈52 tok/s Q3_K_M
across 2 GPUs, temperature 0, coherent output).
Why only two levels
The FP8 source stores routed experts with 128×128 block scales, and the converter repacks every expert straight to MXFP4 — 94% of the weight. So MXFP4 comes out smaller than the source and Q4_K_M / Q5_K_M / Q6_K would all be larger (they re-inflate 4-bit experts). Only Q3 and below actually shrink. Q2_K was measured on the sister 0731 model and deliberately not shipped: it kept fluent prose and a perfect 0% refusal rate while MMLU collapsed to chance (23%). Refusal rate is no guide to quantization damage; only a capability benchmark catches the cliff.
Requirements
You need a llama.cpp with DeepSeek-V4-Flash support (the deepseek_v4 architecture, hash-routed MoE,
hyper-connections). It is recent and still moving; build from source if your version predates it. A
symptom of a too-old or mismatched build is done_getting_tensors: wrong number of tensors at load.
imatrix is not usable on this family (imatrix size 32768 != tensor size 4096), so these are plain
quantizations. MTP is dropped by the converter (conversion v0), so there is no speculative draft head
in the GGUF.
Usage
# follows the split set from the first shard automatically
llama-cli -m DeepSeek-V4-Flash-Vision-Exp-Abliterated-MXFP4-00001-of-00004.gguf \
-ngl 99 -c 8192 -p "..."
License
MIT, inherited from the base model. See the parent repo for the full disclaimer, intended-use notes, and the abliteration + evaluation details.
Support / 打賞
USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888