gemma-4-E4B-it-heretic

A decensored variant of google/gemma-4-E4B-it, produced with Heretic v1.4.0 (directional ablation / "abliteration"). Google's Gemma 4 E4B is an 8B-class multimodal MoE that takes text, image, video, and audio and generates text, with a 128K context and configurable thinking modes; refusal behaviour is suppressed here via targeted weight edits to the attention output and MLP down-projections rather than fine-tuning, so the model's multimodal perception and reasoning are left largely intact.

Who this is for: developers who want Gemma 4's multimodal range — image, video, and audio input — without the alignment refusals, on hardware they already own. E4B is the sweet spot in the family: it runs on-device on a laptop, handles 128K context, supports thinking modes, and here answers directly instead of declining. Best for local document/UI/screenshot analysis, audio transcription and understanding, and multilingual assistant work over 140 languages.

Runs on your gaming PC

Full GGUF ladder included - pick the quant that fits your card:

Weights only, at this model's native 8B size; add ~1 GB per 32K of context. Because this is a mixture-of-experts model, memory bandwidth matters far more than raw VRAM — a CPU with fast RAM often outruns a slow GPU. OOM? Drop one quant level. Headroom to spare? Go one up.

Abliteration parameters

Trial 176 of a 300-trial Heretic run (seed 2082950097). direction_index was selected per layer.

Performance

Refusals on the harmful evaluation set drop from 100/100 to 25/100 — the most complete refusal removal in this batch, and the base model refused essentially everything tested. KL divergence of 0.1215 is a moderate-fidelity edit: Gemma 4's safety tuning is broad and consistent, so Heretic had to move further to reach this refusal level than it did on the Granite or Qwen siblings. Expect more drift in tone and formatting than a low-KL model, and spot-check the model's behaviour on your own prompts before relying on it.

Why abliteration instead of fine-tuning

Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the Heretic repo and the original abliteration writeup for the mechanism.

Made with ❤️ by RACER IS OP (saidutta69) — follow for more uncensored models

Files

Safetensors

BF16. The reproduce/ directory carries the full Heretic recipe — config.toml, requirements.txt, the Optuna study journal, and SHA-256 sums — so this exact model can be regenerated bit-for-bit. Reproduce it with heretic --reproduce reproduce/reproduce.json.

GGUF quantizations

Full quantization set (14 quants + F16) produced with llama.cpp.

Gemma 4 architecture (gemma4) - loads natively in llama.cpp / Ollama / LM Studio / Jan.

Run llama serve -hf saidutta69/gemma-4-E4B-it-heretic to pull the default quant.

Quickstart

# llama.cpp
llama serve -hf saidutta69/gemma-4-E4B-it-heretic

# transformers
from transformers import AutoProcessor, AutoModelForImageTextToText

model_name = "saidutta69/gemma-4-E4B-it-heretic"
model = AutoModelForImageTextToText.from_pretrained(model_name, dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained(model_name)

messages = [{"role": "user", "content": [
    {"type": "image", "image": "https://example.com/photo.jpg"},
    {"type": "text", "text": "Describe what is in this image and transcribe any text you can read."},
]}]
inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,
                                        return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=1024)
print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Also runnable via Ollama, LM Studio, Jan, vLLM, SGLang.

Thinking modes

Gemma 4 is built as a reasoner with configurable thinking. Toggle the reasoning block through the chat template — full thinking for hard problems, non-thinking when you want latency:

# full thinking (default)
processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

# skip the reasoning block
processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False,
                              enable_thinking=False)

Strip the reasoning block from the output before displaying it to users if you enabled thinking.

Model details

About the Gemma 4 family

Gemma 4 is a family of open multimodal models from Google DeepMind with text, image, and video input (audio on E2B, E4B, and 12B) and text output. The family spans five sizes — E2B, E4B, 12B, 26B A4B, and 31B — mixing dense and mixture-of-experts architectures, with a 128K context on the small models and 256K on the medium ones. E4B is the on-device tier: multimodal range and reasoning in a model that fits on a laptop.

FAQ

What is gemma-4-E4B-it-heretic?

gemma-4-E4B-it-heretic is a decensored build of google/gemma-4-E4B-it produced with Heretic 1.4.0 (directional ablation). It is the same weights with the refusal direction removed, not a new fine-tune, so its capabilities track the base model.

Is gemma-4-E4B-it-heretic actually uncensored?

Yes. On Heretic's 100-prompt refusal harness it drops from 100/100 to 25/100 at KL divergence 0.1215 from the base model. These are measured values, not a claim; reproduction data is in reproduce/. This edit is conservative: only 75 of 100 refusals cleared, so read KL and refusal count together.

How do I run gemma-4-E4B-it-heretic locally?

Text only: llama serve -hf saidutta69/gemma-4-E4B-it-heretic:Q4_K_M (one of 15 GGUF quants). The GGUFs drop the vision tower; use the BF16 safetensors plus AutoProcessor for image input.

Responsible use

Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. You are responsible for how you deploy it — don't put this behind an unmoderated public-facing endpoint serving third parties. It inherits Gemma 4's factual limitations and biases; abliteration removes refusal directions, it doesn't add capability or judgment.

License

Inherits the Gemma 4 license from the base model. Upstream lists it as Apache 2.0 with the Gemma terms of use — read the linked terms before commercial deployment.

Related

• google/gemma-4-E4B-it — the base model
• Gemma 4 collection — the rest of the family
• Gemma 4 technical report
• RACER IS OP — Heretic Models — full collection

Abliteration hardware

Recorded in reproduce/reproduce.json:

• GPU: NVIDIA GeForce RTX 5090 (31.36 GB VRAM)
• CPU: AMD Ryzen 9 9950X 16-Core Processor
• OS: Linux-6.6.0-hiveos-x86_64-with-glibc2.35
• CUDA / driver: CUDA 13.0, driver 580.173.02
• Software: PyTorch 2.14.1+cu130, Heretic 1.4.0

Heretic notes that KL-divergence and refusal numbers can vary slightly with hardware and library versions.