Gemma 4 12B (QAT-Q4_0 unquantized) — Heretic decensored — GGUF
GGUF quantizations of igorls/gemma-4-12B-it-qat-q4_0-unquantized-heretic, a decensored ("abliterated") version of Google's Gemma 4 12B QAT-Q4_0 unquantized checkpoint, produced with Heretic v1.3.0.
| Quant | Size | Notes |
|---|---|---|
Q4_0 |
6.5 GB | Recommended. QAT-matched: the base was quantization-aware-trained for the Q4_0 grid, so this 4-bit quant best preserves the original quality |
Q4_K_M |
6.9 GB | Standard K-quant 4-bit alternative |
Q8_0 |
11.8 GB | Near-lossless |
[!NOTE] This model derives from Google's QAT-Q4_0 checkpoint. Quantization-aware training was calibrated specifically for the
Q4_0grid, soQ4_0is the quant that realizes the QAT advantage (≈bf16 quality at 4-bit).Q4_K_Mis a normal K-quant of the same weights and does not specifically leverage QAT.
v1.1 — thinking-mode fix
Gemma 4 is a thinking model: its refusal decision forms inside the chain-of-thought. The first release was abliterated/evaluated with thinking disabled, so it still refused once thinking was on (the default in Ollama / llama.cpp). v1.1 decensors the model with thinking enabled — the way it's actually used.
Performance
| Metric | This model (v1.1) | Original |
|---|---|---|
| KL divergence | 0.32 | 0 (by definition) |
| Refusals, thinking on (adversarial harmful set) | ~22% | ~99% |
KL is high relative to a typical abliteration — that is the cost of suppressing refusal through the reasoning trajectory. The model complies with the vast majority of requests in normal use; a small fraction of extreme prompts may still be refused, and a stronger v2 is in progress.
Run with llama.cpp
llama-cli -hf igorls/gemma-4-12B-it-qat-q4_0-unquantized-heretic-GGUF:Q4_0
Run with Ollama
ollama run hf.co/igorls/gemma-4-12B-it-qat-q4_0-unquantized-heretic-GGUF:Q4_0
Requires a llama.cpp / Ollama build with Gemma 4 (gemma4) support.
Disclaimer
This is an abliterated model: its built-in safety guardrails and refusal behavior have been deliberately removed. As a result it will attempt to answer essentially any prompt and can produce content that is offensive, inaccurate, explicit, or otherwise harmful — content the original Gemma 4 would have refused.
- No safety alignment. Do not rely on it to refuse unsafe requests or to self-moderate. Apply your own filtering, guardrails, and human review before any production or user-facing use.
- You are solely responsible for how you use this model and for complying with all applicable laws and with the base model's Gemma 4 license (Apache 2.0).
- Intended for adults (18+), for research, evaluation, and lawful creative use where permitted.
- Provided as-is, without warranty of any kind. The author accepts no liability for any output produced or any use of this model.