Sign up

audreyt/DeepSeek-V4.1-Flash-Abliterated-GGUF

🤗 Hugging Face sourcemit731 GBGGUF✓ Checksums witnessedupdated today
No torrent yet →

DeepSeek-V4.1-Flash-Abliterated-Q2-GGUF

DwarfStar Q2 GGUF of the community-abliterated DeepSeek V4.1 Flash checkpoint (s-zaizen/DeepSeek-V4.1-Flash-Abliterated, itself abliterated from deepseek-ai/DeepSeek-V4.1-Flash with Heretic; refusal keyword rate reported there as 97/100 → 24/100).

Two files: ...-imatrix.gguf (recommended) and the original DeepSeek-V4.1-Flash-Abliterated-Q2.gguf bootstrap build (kept for reference; the imatrix build supersedes it).

Recipe

Converted with deepseek41_quantize.py (DwarfStar ds4 repo) using the project's C quantizers, --quant q2, imatrix-calibrated routed experts (16-prompt / 26K-token calibration collected on the abliterated Q2):

  • Routed MoE experts: IQ2_XXS gate/up, Q2_K down (imatrix-guided on the stock recipe; bootstrap here — an imatrix-calibrated v2 may follow)
  • Attention, shared experts, projections, output: Q8_0 / F16
  • Engram FP8 rows packed losslessly at end of file (read from disk as needed, never resident)

deepseek41_validate_gguf.py layout audit: PASS (1046 tensors).

Quality vs stock Q2

Target-token NLL on 100 tracked official V4.1 Flash continuations (deepseek-v4.1-flash-20260910-general, ctx 4096, SSD streaming), same harness for both files:

file mean NLL median NLL first-token match top-1 agreement
stock DeepSeek-V4.1-Flash-Q2 0.346 0.313 78/100 0.901
bootstrap build 0.395 0.395 71/100 0.888
imatrix build (this file) 0.359 0.340 80/100 0.904

Essentially at parity with stock: the remaining ~0.01 NLL is plausibly the ablation itself. First-token match and top-1 agreement beat stock within noise.

Use

Built for DwarfStar ds4 (docs/MODELS.md "DeepSeek V4.1 Flash"). It is not a general GGUF: the tensor layout, quantization mix, and Engram packing are ds4-specific.

./download_model.sh ds41f-q2  # stock weights instead (341 GiB)
./ds4 -m DeepSeek-V4.1-Flash-Abliterated-Q2.gguf --ssd-streaming --ctx 32768

On one 128 GB Mac, use --ssd-streaming (152 GiB main weights; Engram tables stay on disk). Needs a fast local SSD.

License

Base weights © DeepSeek, MIT (retained via the source checkpoint). Conversion tooling per the ds4 repo license.