Sign up

petruhonk/Qwen3.8-9B-Distill-uncensored-heretic-GGUF

🤗 Hugging Face sourcetext-generationapache-2.028 GBGGUF✓ Checksums witnessedupdated today
No torrent yet →

Qwen3.8-9B-Distill-Heretic-MTP

Decensored empero-ai/Qwen3.8-9B-Distill, made with Heretic v1.4.0.

Safetensors version: petruhonk/Qwen3.8-9B-Distill-uncensored-heretic

The MTP head is kept. That is the point of this repository. The existing abliteration of this model on the Hub ships 427 tensors and 32 layers — the blk.32.nextn.* block is gone, and with it llama.cpp's --spec-type draft-mtp. This conversion keeps all 442 tensors and 33 blocks, so speculative decoding works: measured ~73 tok/s generation with it (draft acceptance 0.73), on an RTX 3060.

Performance

Metric This model Original
Refusals 6/100 100/100
KL divergence 0.0306 0 (by definition)

Refusals are measured by Heretic on mlabonne/harmful_behaviors; lower is better.

Why another one

The published abliteration of this model (rohit267/Qwen3.8-9B-heretic-uncensored) reports 98/100 refusals against the original's 100/100 — two refusals removed, at KL 0.0008. It barely changed the model, and it dropped the MTP head. This run optimised further and preserved the head.

Usage

llama-server -m Qwen3.8-9B-Distill-Heretic-Q8_0.gguf \
  -c 32768 --parallel 1 -ctk q8_0 -ctv q8_0 -fa on -ngl 99 \
  --jinja --spec-type draft-mtp --spec-draft-n-max 3

--jinja matters: the chat template inside the GGUF carries tools and <tool_call>, so function calling works and the model can drive an agent.

KV cache in q8_0 is not a quality choice but a memory one: f16 at 32k context needs 4.12 GiB, which does not fit beside the weights on a 12 GiB card. With more VRAM, drop to -ctk f16 -ctv f16.

The model is multimodal (Qwen3_5ForConditionalGeneration, vision + video); this repository contains the text weights only. An mmproj file is needed for images.

Provenance

Abliteration and conversion done on a single RTX 3060 12GB: Heretic with --quantization bnb_4bit, then convert_hf_to_gguf.py (without --no-mtp, so the nextn block is carried over), then llama-quantize to Q8_0.