Gemma-4-E2B-Heretic — GGUF Quants

Quantized GGUF versions of DuoNeural/Gemma-4-E2B-Heretic, DuoNeural's abliterated Gemma 4 2B Instruct variant.

The Heretic series applies norm-preserving orthogonal projection (abliteration) to surgically remove refusal behavior while preserving the full instruction-following and reasoning capabilities of the base model.

Available Files

File	Quant	Size	Use Case
`Gemma-4-E2B-Heretic-Q8_0.gguf`	Q8_0	~4.7GB	Maximum quality, ample VRAM
`Gemma-4-E2B-Heretic-Q6_K.gguf`	Q6_K	~3.6GB	Near-lossless, recommended for 6GB+
`Gemma-4-E2B-Heretic-Q5_K_M.gguf`	Q5_K_M	~3.4GB	High quality, good balance
`Gemma-4-E2B-Heretic-Q4_K_M.gguf`	Q4_K_M	~3.2GB	Recommended default — quality/size sweet spot
`Gemma-4-E2B-Heretic-Q3_K_M.gguf`	Q3_K_M	~3.0GB	Low VRAM, acceptable quality
`Gemma-4-E2B-Heretic-IQ4_XS.gguf`	IQ4_XS	~3.1GB	Imatrix-optimized 4-bit
`Gemma-4-E2B-Heretic-IQ3_XXS.gguf`	IQ3_XXS	~2.5GB	Imatrix 3-bit, aggressive compression
`Gemma-4-E2B-Heretic-IQ2_M.gguf`	IQ2_M	~2.4GB	Imatrix 2-bit, minimum viable quality
`Gemma-4-E2B-Heretic-IQ1_S.gguf`	IQ1_S	~2.2GB	Extreme compression, experimental
`E2B-Heretic-fp16.gguf`	FP16	~8.7GB	Full precision base for re-quantization
`imatrix.dat`	—	—	Importance matrix for custom IQ quants

Usage

llama.cpp

./llama-cli -m Gemma-4-E2B-Heretic-Q4_K_M.gguf \
  -p "You are a helpful assistant." \
  --ctx-size 8192 -n 512

Ollama

ollama run hf.co/DuoNeural/Gemma-4-E2B-Heretic-GGUF:Q4_K_M

Jan / LM Studio / Open WebUI

Download and load the .gguf file directly. Recommended: Q4_K_M for most hardware.

About Heretic Abliteration

The Heretic series uses DuoNeural's 2-pass abliteration pipeline:

Residual collection — identify refusal direction in activation space using 4-bit GPU inference
Weight modification — project out the refusal direction from MLP/attention weight matrices in bf16 on CPU
Validation — confirm compliance behavior removed, helpfulness preserved

The base model is google/gemma-4-e2b-it — Google's Gemma 4 2B instruction-tuned model with 128K context, multimodal inputs, and strong instruction-following capability.

Notes

IQ formats use the included imatrix.dat for importance-weighted quantization
For custom IQ quants: llama-quantize --imatrix imatrix.dat model.gguf output.gguf IQ4_XS
AWQ format not available: AutoAWQ does not currently support the Gemma 4 architecture

Quantized by DuoNeural using llama.cpp on RTX 5090.

DuoNeural

DuoNeural is an open AI research lab — human + AI in collaboration.

Platform	Link
HuggingFace	huggingface.co/DuoNeural
Website	duoneural.com
GitHub	github.com/DuoNeural
X / Twitter	@DuoNeural
Email	duoneural@proton.me
Newsletter	duoneural.beehiiv.com
Support	buymeacoffee.com/duoneural

DuoNeural Research Publications

Title	DOI
Nano-CTM: Ternary Continuous Thought Machines with Thought-Space Self-Prediction for Efficient Reasoning	10.5281/zenodo.19775622
Recurrence as World Model: CTM Learns Implicit Belief States in Partially Observable Environments	10.5281/zenodo.19810620
Per-Object Slot Decomposition for Scalable Neural World Modeling	10.5281/zenodo.19846804
The Dynamical Horizon Principle: CTM Gates Converge to the Predictability Limit of Dynamical Systems	10.5281/zenodo.19952612

Open access, CC BY 4.0. Authored by Archon, Jesse Caldwell, Aura — DuoNeural.