granite-4.1-3b-heretic

A decensored variant of ibm-granite/granite-4.1-3b, produced with Heretic v1.4.0 (directional ablation / "abliteration"). IBM's compact Granite 4.1 instruct model keeps its native tool calling, RAG grounding, Fill-In-the-Middle code completion, and 128K context; refusal behaviour is suppressed via targeted weight edits to the attention output and MLP down-projections rather than fine-tuning, so the base model's instruction-following and agentic ability are left largely intact.

Who this is for: developers who want an agent that actually uses its tools, without the safety theatre. Granite 4.1 3B is genuinely capable for its size - IFEval 82.30 for instruction following, BFCL v3 60.80 for tool calling, GSM8K 86.88 and HumanEval 81.71 - which is a rare combination at 3B. Refusals drop from 89/100 to 3/100. Use it for local tool-calling agents, document-grounded RAG, structured extraction, and multilingual chat on CPU or a low-VRAM GPU. If you want the same model with an explicit <think> reasoning block, reach for granite-4.2-3b-heretic instead - 4.1 is the non-reasoning sibling.

Runs on your gaming PC

Full GGUF ladder included - pick the quant that fits your card:

Weights only, at this model's 3.40B native size; add ~1 GB for context. OOM? Drop one quant level. Headroom to spare? Go one up.

Abliteration parameters

Trial 161 of a 200-trial Heretic run (seed 3455290582).

Performance

Refusals on the harmful evaluation set drop from 89/100 to 3/100 - the alignment tuning is essentially gone. Be aware of the trade: a KL divergence of 0.0852 is one of the looser edits in this collection, roughly 9x the tightest ones. Heretic targets KL 0.01, and this run settled well above that, so expect noticeably more drift in phrasing and tone than a low-KL abliteration. The tool-call format, chat template, and instruction-following survive intact, but if you depend on exact output behaviour, run your own evals before committing.

Base model benchmarks

Abliteration does not add capability - these are the base numbers, from IBM's Granite 4.1 eval:

Why abliteration instead of fine-tuning

Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the Heretic repo and the original abliteration writeup for the mechanism.

Made with ❤️ by RACER IS OP (saidutta69) — follow for more uncensored models

Files

Safetensors

BF16, 3,402,836,480 parameters, with tied input/output embeddings. The reproduce/ directory carries the full Heretic recipe - config.toml, requirements.txt, the Optuna study journal, and SHA-256 sums - so this exact model can be regenerated bit-for-bit. Reproduce it with heretic --reproduce reproduce/reproduce.json.

GGUF quantizations

Full quantization set (14 quants + F16) produced with llama.cpp.

Granite architecture (granite, 362 tensors) - loads natively in llama.cpp / Ollama / LM Studio / Jan.

Run llama serve -hf saidutta69/granite-4.1-3b-heretic to pull the default quant.

Quickstart

# llama.cpp
llama serve -hf saidutta69/granite-4.1-3b-heretic

# transformers (requires transformers>=5.0.0 - this repo uses the v5 config/tokenizer format)
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "saidutta69/granite-4.1-3b-heretic"
model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

messages = [{"role": "user", "content": "A train leaves at 14:05 travelling 80 km/h. A car leaves at 14:15 travelling 120 km/h on the same track. When does the car catch the train? Show your work."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=1024)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Also runnable via Ollama, LM Studio, Jan, vLLM, SGLang.

Chat template

Granite 4.1 uses an explicit role-tag format. There is no <think> block in this generation - if you need visible chain-of-thought, use the 4.2 model.

<|start_of_role|>user<|end_of_role|>What is the capital of France?<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|>Paris.<|end_of_text|>

Tool calling

Pass OpenAI-style function definitions as tools and let the template do the rest - it injects the <tools> system block and the tool-call instruction for you. Do not hand-roll the format.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

text = tokenizer.apply_chat_template(messages, tools=tools, add_generation_prompt=True, tokenize=False)

The model emits calls as JSON inside <tool_call> tags; feed results back under a tool role and the template wraps them in <tool_response> tags for you.

Document grounding (RAG)

The template also accepts a documents argument. Pass a list of strings and it emits a <documents> block plus an instruction to answer strictly from those documents - no retrieval framework or prompt engineering required.

text = tokenizer.apply_chat_template(messages, documents=[chunk1, chunk2],
                                     add_generation_prompt=True, tokenize=False)

Model details

FAQ

What is granite-4.1-3b-heretic?

granite-4.1-3b-heretic is a decensored build of ibm-granite/granite-4.1-3b produced with Heretic 1.4.0 (directional ablation). It is the same weights with the refusal direction removed, not a new fine-tune, so its capabilities track the base model.

Is granite-4.1-3b-heretic actually uncensored?

Yes. On Heretic's 100-prompt refusal harness it drops from 89/100 to 3/100 at KL divergence 0.0852 from the base model. These are measured values, not a claim; reproduction data is in reproduce/.

How do I run granite-4.1-3b-heretic locally?

llama serve -hf saidutta69/granite-4.1-3b-heretic:Q4_K_M pulls the default quant; the repo ships 15 GGUF quants (see Files).

Responsible use

Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. You are responsible for how you deploy it - don't put this behind an unmoderated public-facing endpoint serving third parties. It inherits Granite 4.1's factual limitations and biases; abliteration removes refusal directions, it doesn't add capability or judgment.

License

Inherits the apache-2.0 license from the base model. IBM ships Granite 4.1 under Apache 2.0 but does not include a LICENSE file in the repo, so the canonical license text is linked instead.

Related

• ibm-granite/granite-4.1-3b - the base model
• saidutta69/granite-4.2-3b-heretic - the same size with a <think> reasoning block
• saidutta69/granite-4.1-8b-heretic - the 8B sibling
• saidutta69/Parable-Granite-4.1-3B-Claude-Fable-5-heretic - a different 3B fine-tune of the same base, tuned for agentic coding
• Granite 4.1 technical blog
• RACER IS OP - Heretic Models - full collection

Abliteration hardware

Recorded in reproduce/reproduce.json:

• GPU: NVIDIA GeForce RTX 4090 (23.52 GB VRAM)
• CPU: AMD EPYC 7402 24-Core Processor
• OS: Linux-6.8.0-142-generic-x86_64-with-glibc2.35
• CUDA / driver: CUDA 13.0, driver 595.91.07
• Software: PyTorch 2.14.1+cu130, Heretic 1.4.0

Heretic notes that KL-divergence and refusal numbers can vary slightly with hardware and library versions.