⚡ Qwen3.8-27B-abliterated (UD-Q6_K_L · Q8 ablated writers)

Uncensored hybrid-attention GGUF by KatterMobile (Kaden)

❤️ Please leave a LIKE / HEART on this Hugging Face repository! If you find this release useful, hitting the heart button at the top of this page makes a huge difference and helps independent developers like me keep releasing frontier open-source models.

---

☕ A Personal Note from KatterMobile

"Hello! I'm KatterMobile (real name Kaden). I reverse-engineered this hybrid Qwen3.8-27B GGUF on a single NVIDIA GB10 Grace Blackwell workstation — mapped every tensor, extracted per-layer refusal directions with llama.cpp itself, and baked the ablation directly into the Unsloth UD quantization without a single byte of collateral damage to the other 798 tensors. I am an independent researcher without corporate backing. If this release is useful to you, please consider supporting me on Ko-fi — anything helps cover compute and hosting. Thank you!"

☕ Support on Ko-fi: https://ko-fi.com/kattermobile 📬 Support & Inquiries: support@kattermobile.com

---

🌟 Model Overview

Qwen3.8-27B-abliterated-UD-Q6_K_L-Q8 is an uncensored build of unsloth/Qwen3.8-27B-GGUF UD-Q6_K_L (24.18 GB), itself a dynamic quantization of Qwen/Qwen3.8-27B.

The refusal direction was removed in float32 inside the GGUF itself, then only the 68 edited tensors were re-quantized — with Unsloth's own imatrix and the original per-tensor type recipe. Every other tensor is byte-identical to the upstream UD file.

This is not a LoRA, not a fine-tune, and not a re-quantized safetensors dump: it is a surgical weight edit of the exact artifact you run.

📊 At a Glance

🧠 How the Abliteration Was Done

1. Reverse-engineered the GGUF. 65 blocks: 48 gated linear-attention (SSM) blocks (attn_qkv 5120→10240, attn_gate 5120→6144, ssm_out 6144→5120) and full attention every 4th block (attn_q 5120→12288 with fused gate, attn_k/v 1024, attn_output 6144→5120).
2. Extracted per-layer refusal directions with llama-cvector-generator from 20 paired harmful/harmless prompts. Each direction.N is the mean residual-stream difference at the output of block N−1.
3. Orthogonalized the residual writers in float32: W ← W − s · d (dᵀW) / (dᵀd) with s = 1.35. This removes the direction from everything those matrices write into the residual stream.
4. Re-quantized only the edited tensors with llama-quantize, the Unsloth imatrix (imatrix_unsloth.gguf) and an anchored per-tensor type recipe. Untouched tensors are copied with quantize = cur_type != new_type, i.e. bit-exact.
5. Verified the surgery: 798/866 tensors byte-identical, and the projection along the refusal direction driven from ≈1.5 % to ≈0.55 % with the sign inverted (over-ablation), so quantization noise can no longer restore it.

Why the ablated tensors are Q8_0

A naive scale-1.0 edit re-quantized back to Q6_K measures like this:

Quantization noise re-injects ~25 % of the removed direction. Storing the 68 edited tensors at Q8_0 and over-ablating at 1.35 is what actually closes the gap — the same trick huihui uses for its _K_L files.

🆚 How This Differs From Huihui's Release

📈 Measured Results

Smoke evaluation: 20 refusal-inducing prompts, temperature 0, thinking disabled, keyword-scored. Perplexity: wikitext-2 test, 24 × 512 tokens.

[!NOTE] This is a 20-prompt smoke test, not a safety evaluation. Run a few hundred held-out prompts and a capability benchmark before relying on it for anything serious.

📁 Files

🚀 Quick Start (llama.cpp)

llama-cli -m Qwen3.8-27B-abliterated-UD-Q6_K_L-Q8.gguf \
  -ngl 99 -c 8192 --temp 1.0 --top-p 0.95 --top-k 20 \
  -p "Explain why the sky is blue."

llama-server -m Qwen3.8-27B-abliterated-UD-Q6_K_L-Q8.gguf \
  -ngl 99 -c 32768 --port 8080 --reasoning-budget 0

--reasoning-budget 0 disables the thinking block; drop it (or use -1) for full reasoning.

Recommended sampling

🔁 Reproduce It Yourself

# 1. per-layer refusal directions (llama.cpp, no transformers)
llama-cvector-generator -m Qwen3.8-27B-UD-Q6_K_L.gguf -ngl 99 \
  --positive-file harmful.txt --negative-file harmless.txt \
  --method mean -o cvec_mean.gguf

# 2. float32 orthogonalization of the 68 residual writers
python3 abliterate_gguf.py --src Qwen3.8-27B-UD-Q6_K_L.gguf --dst patched_f16.gguf \
  --directions cvec_mean.gguf --layers 18-51 --scale 1.35 --out-type f16

# 3. re-quantize with the original imatrix + per-tensor recipe
llama-quantize --imatrix imatrix_unsloth.gguf \
  --tensor-type-file recipe.txt patched_f16.gguf out.gguf Q6_K 20

⚠️ Usage Warnings

• This model has had its refusal behaviour removed on purpose. It will produce content that the base model declines to produce. That is the point of the release.
• Not suitable for public-facing, underage, or high-security deployments.
• You are responsible for what you generate with it and for complying with local law.
• No warranty of any kind; provided as-is under Apache-2.0.

---

☕ Support independent open-source AI

Ko-fi: https://ko-fi.com/kattermobile · Email: support@kattermobile.com

Built and verified on a single NVIDIA GB10 Grace Blackwell workstation.