⚡ Gemma 4 E4B Heretic QAT — Q4_0 GGUF

Heretic ARA · QAT-Lossless Q4_0 · 4 GB · Mobile-Ready

Q4_0 4.5B Effective Heretic Uncensored 4 GB QAT Weights 📱 Mobile

Uncensored version of Google Gemma 4 E4B IT (QAT), processed with Heretic ARA abliteration. Quantized to Q4_0 — only 4 GB, perfect for mobile and edge devices.

✂️ Heretic ARA Abliteration Parameters

Base: coder3101/heretic-QAT · Heretic v1.2.0 · ARA + Row-Norm

Parameter	Value
start_layer_index	20
end_layer_index	36
preserve_good_behavior_weight	0.5805
steer_bad_behavior_weight	0.0023
overcorrect_relative_weight	0.7764
neighbor_count	13

Metric	Heretic	Original QAT
KL Divergence	0.0065	0 (by definition)
Refusals	5/100	98/100

🏗️ Architecture

Base Model	google/gemma-4-E4B-it
Parameters	4.5B effective (8B with PLE embeddings)
Architecture	Dense with Per-Layer Embeddings (PLE) + multimodal
Layers	42
Hidden Size	2,560
Context Length	128K tokens
Vocabulary	262K, 140+ languages
Modalities	Text + Image + Audio + Video
Quantization	Q4_0 (matching Unsloth UD-Q4_K_XL layout), b9553 llama-quantize

📊 Quantization Details

Format	Q4_0 (QAT weights optimized for this precision)
File Size	4 GB
Effective BPW	4.54 (all weights Q4_0, norms F32)
Tool	llama-quantize (b9553, CUDA 13.3)
QAT Advantage	Q4_0 + QAT weights achieves 98.5% Top-1 vs 90.9% naive Q4_0 (+7.6%)

⚙️ Recommended Sampling Parameters

General	temp=1.0, top_p=0.95, top_k=64
Coding	temp=0.6, top_p=0.95, top_k=64

Use --jinja flag with llama.cpp. Disable thinking: --chat-template-kwargs '{"enable_thinking":false}'.

📝 Usage

Only 4 GB — runs on any modern phone with 6GB+ RAM. Compatible with llama.cpp, LM Studio, Jan, MLC LLM, and mobile GGUF runtimes.

llama-server \
  -m gemma-4-E4B-it-heretic-QAT-UD-Q4_K_XL.gguf \
  --jinja -ngl 99 -c 4096 \
  --port 8001

🔗 Credits

Heretic Abliteration: coder3101 · Heretic v1.2.0 ARA + Row-Norm
QAT Weights: Google Gemma 4 E4B IT
Quantization Recipe: Unsloth UD-Q4_K_XL (Q4_0 layout)
Quantization Tool: llama.cpp b9553 · GitHub
Original Model: Google Gemma 4 E4B IT

SC117/gemma-4-E4B-it-heretic-QAT-GGUF

⚡ Gemma 4 E4B Heretic QAT — Q4_0 GGUF

Model discussions