DeepSeek-R1-0528-Qwen3-8B-Heretic GGUF

GGUF quantizations of richardyoung/DeepSeek-R1-0528-Qwen3-8B-heretic, an abliterated build of deepseek-ai/DeepSeek-R1-0528-Qwen3-8B (DeepSeek's R1-0528 reasoning distilled into Qwen3-8B: a reasoning model that emits chains before answering) made with Heretic.

Abliteration Results

Metric	Value
Base Refusals	89/100
Abliterated Refusals	4/100
Refusal Reduction	96%
KL Divergence	0.041

The very low KL divergence (0.041, far below the 0.5 “damage” threshold) means the model retains essentially all of its original capabilities and coherence.

Quick Start

With Ollama

ollama run richardyoung/deepseek-r1-0528-qwen3-8b-heretic
# or straight from Hugging Face
ollama run hf.co/richardyoung/DeepSeek-R1-0528-Qwen3-8B-heretic-GGUF

With llama.cpp

huggingface-cli download richardyoung/DeepSeek-R1-0528-Qwen3-8B-heretic-GGUF \
    --include "*Q4_K_M*" --local-dir ./models

./llama-cli -m ./models/*Q4_K_M*.gguf -ngl 99

With Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="richardyoung/DeepSeek-R1-0528-Qwen3-8B-heretic-GGUF",
    filename="*Q4_K_M*",
    n_gpu_layers=-1,
)

output = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain abliteration in simple terms."}]
)
print(output["choices"][0]["message"]["content"])

Available Quantizations

Quantization	Size	Use Case
IQ3_M	3.9 GB	Smallest
IQ4_XS	4.6 GB	Great quality-size balance
Q4_K_M	5.0 GB	Recommended — good balance
Q5_K_M	5.9 GB	Higher quality
Q6_K	6.7 GB	Very high quality
Q8_0	8.7 GB	Near-lossless
DeepSeek-R1-0528-Qwen3-8B-heretic-imatrix.gguf	–	Importance matrix used for the IQ and Q4_K_M/Q5_K_M quants (calibration_datav3)

Quant Quality

Each quant compared with the F16 model using llama-perplexity --kl-divergence on the wikitext-2 test set (40 × 512-token chunks). KL divergence measures how far the quant's next-token probabilities drift from F16; lower is better. Top-token match is how often the quant picks the same next token as F16. As a rough guide, a KLD under 0.01 is hard to tell from F16, and from about 0.05 differences start to show in longer outputs.

Quant	Size	KL divergence	Top-token match	Perplexity (F16: 9.85)
IQ3_M	3.9 GB	0.2915	77.3%	10.86
IQ4_XS	4.6 GB	0.1173	85.0%	10.17
Q4_K_M	5.0 GB	0.0961	86.1%	10.13
Q5_K_M	5.9 GB	0.0467	89.8%	10.00
Q6_K	6.7 GB	0.0366	90.9%	10.04
Q8_0	8.7 GB	0.0052	96.5%	9.88

What is Abliteration?

Abliteration removes the "refusal direction" from a model's residual stream — a surgical modification that disables safety refusals without retraining. See Refusal in Language Models Is Mediated by a Single Direction. Heretic automates it: it searches ablation parameters with an optimizer (200 trials here) and keeps the trials that best trade off fewer refusals against KL divergence from the original model.

Intended Use

Research, creative writing, education on alignment techniques, and unrestricted local inference. This model will comply with requests the original refuses; use responsibly and in accordance with applicable laws. License: MIT, inherited from the base model.

Other Models by richardyoung

New Heretic models (reproducible, listed in the Heretic Grimoire): LFM2.5-1.2B | Granite-4.2-3B | Granite-4.2-8B | Olmo-3-7B-Think | Phi-4-mini | DeepSeek-R1-0528-Qwen3-8B | Agents-A1-4B | FrogNano-4B | Muse-Glimmer-30B | Hemmingway-1
Abliterated/Uncensored models: Qwen2.5-7B | Qwen3-14B | DeepSeek-R1-32B | Qwen3-8B
OCR & Vision: MiniCPM-V-4.6 GGUF | dots.mocr GGUF | Unlimited-OCR GGUF | olmOCR GGUF | LightOnOCR-3-4B GGUF | WeVisDoc-4B GGUF | Xiaomi-OCR-0 GGUF
MLX quantizations (Apple Silicon): Kimi-K2 series | olmOCR MLX
Healthcare/Medical: Synthea 575K patients dataset | CardioEmbed
Research: LLM Instruction-Following Evaluation (arxiv:2510.18892)

Built & maintained by Richard Young · DeepNeuro

richardyoung/DeepSeek-R1-0528-Qwen3-8B-heretic-GGUF