Qwen3-4B-Thinking-heretic

A decensored variant of Qwen/Qwen3-4B-Thinking-2507, produced with Heretic v1.4.0 (directional ablation / "abliteration"). Qwen's always-thinks 4B reasoning model keeps its full <think> chain-of-thought, tool calling, and 262K context; refusal behaviour is suppressed via targeted weight edits to the attention output and MLP down-projections rather than fine-tuning, so the base model's reasoning is left essentially untouched.

Who this is for: developers who want the strongest small reasoning model they can run locally, without the refusals. The base scores AIME25 81.3 and HMMT25 55.5 on maths reasoning, IFEval 87.4 on instruction following, LiveCodeBench v6 55.2 on code, and BFCL-v3 71.2 on tool calling - a rare combination at 4B. Refusals drop from 98/100 to 4/100. Use it for local agents, math and code work that needs visible reasoning, and multilingual chat between English and Chinese on CPU or a single consumer GPU.

If you want the same model with no reasoning block and faster token throughput, use Qwen3-4B-Instruct-2507-heretic instead - it is the non-thinking sibling, and it also supports an enable_thinking toggle that this one deliberately does not have.

Runs on your gaming PC

Full default quant set - pick the one that fits your card:

Weights only, at this model's 4.02B native size. Thinking mode writes a lot of tokens before it answers, so budget real KV-cache headroom on top - a reasoning trace of a few thousand tokens costs more than the weights do on a long conversation. OOM? Drop one quant level.

Abliteration parameters

Trial 88 of a 200-trial Heretic run (seed 263519039).

Performance

This is about as clean as abliteration gets. Refusals on the harmful evaluation set drop from 98/100 to 4/100 - the base model was almost fully refusal-tuned, and nearly all of it is gone - while the KL divergence of 0.0011 is the tightest in this collection, meaning the weight edit barely perturbed the network. Both at once is the point: the refusal behaviour here was a cleanly separable direction, so removing it cost almost nothing.

Worth saying plainly, because it is not automatic: a low KL number on its own does not mean a good abliteration. Another model in this collection, FrogNano-4B-heretic, has a similarly tight KL of 0.0046 and still refuses 71/100 times - there the optimiser found a direction it barely dared apply, so it preserved behaviour and preserved most refusals. Judge the pair of numbers together, never the KL alone.

Base model benchmarks

Abliteration does not add capability - these are the base numbers, from Qwen's own eval table (column Qwen3-4B-Thinking-2507). No post-abliteration eval has been run.

Why abliteration instead of fine-tuning

Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the Heretic repo and the original abliteration writeup for the mechanism.

Made with ❤️ by RACER IS OP (saidutta69) — follow for more uncensored models

Files

Safetensors

BF16, 4,022,468,096 parameters, with tied input/output embeddings. The reproduce/ directory carries the full Heretic recipe - config.toml, requirements.txt, the Optuna study journal, and SHA-256 sums - so this exact model can be regenerated bit-for-bit. Reproduce it with heretic --reproduce reproduce/reproduce.json.

GGUF quantizations

Default set (F16 + 4 quants) produced with llama.cpp.

Qwen3 dense architecture (qwen3, 398 tensors, 36 layers) - loads natively in llama.cpp / Ollama / LM Studio / Jan.

Run llama serve -hf saidutta69/Qwen3-4B-Thinking-heretic to pull the default quant.

Quickstart

# llama.cpp
llama serve -hf saidutta69/Qwen3-4B-Thinking-heretic

# transformers (requires transformers>=5.0.0)
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "saidutta69/Qwen3-4B-Thinking-heretic"
model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

messages = [{"role": "user", "content": "A train leaves at 14:05 travelling 80 km/h. A car leaves at 14:15 travelling 120 km/h on the same track. When does the car catch the train? Show your work."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, temperature=0.6, top_p=0.95, top_k=20, max_new_tokens=4096)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Thinking mode - what is different here

This checkpoint always thinks. There is no enable_thinking flag to turn it off - the chat template appends <think> to the generation prompt unconditionally, because the Thinking-2507 release dropped the hybrid thinking/non-thinking toggle that Qwen3-4B-Instruct-2507-heretic still exposes. If you need to disable thinking, use that model instead.

Parsing gotcha: it is normal for the output to contain only </think> with no opening <think> tag, because the template already supplied the opening half. Strip reasoning with a regex that tolerates a missing opening tag:

import re
def split_think(text):
    m = re.search(r"</think>(.*)", text, re.S)
    return (m.group(1).strip() if m else ""), (re.sub(r"^.*?</think>", "", text, flags=re.S).strip()
                                               if "</think>" in text else "")

For multi-turn conversations, pass the previous turn's reasoning back through reasoning_content on the assistant message; the template reconstructs the <think>...</think> block for you.

Sampling parameters

Qwen's recommended settings for thinking mode, which the repo's generation_config.json already sets, so generate() picks them up with no arguments:

temperature=0.6, top_p=0.95, top_k=20, do_sample=True

Stop tokens are <|im_end|> (151645) and <|endoftext|> (151643) - both are already configured. Do not pass enable_thinking; this template does not accept it.

Tool calling

Pass OpenAI-style function definitions as tools. The template injects the <tools> system block and instructs the model to emit calls as JSON inside <tool_call> tags; feed results back under a tool role and the template wraps them in <tool_response> for you. Thinking happens before the tool call, so budget for a reasoning trace per turn.

Model details

FAQ

What is Qwen3-4B-Thinking-heretic?

Qwen3-4B-Thinking-heretic is a decensored build of Qwen/Qwen3-4B-Thinking-2507 produced with Heretic 1.4.0 (directional ablation). It is the same weights with the refusal direction removed, not a new fine-tune, so its capabilities track the base model.

Is Qwen3-4B-Thinking-heretic actually uncensored?

Yes. On Heretic's 100-prompt refusal harness it drops from 98/100 to 4/100 at KL divergence 0.0011 from the base model. These are measured values, not a claim; reproduction data is in reproduce/.

How do I run Qwen3-4B-Thinking-heretic locally?

llama serve -hf saidutta69/Qwen3-4B-Thinking-heretic:Q4_K_M pulls the default quant; the repo ships 5 GGUF quants (see Files).

Responsible use

Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. You are responsible for how you deploy it - don't put this behind an unmoderated public-facing endpoint serving third parties. It inherits Qwen3's factual limitations and biases; abliteration removes refusal directions, it doesn't add capability or judgment.

Because thinking mode narrates its reasoning before acting, be aware that the visible chain-of-thought is a reasoning artefact, not a reliable audit trail - a model can produce a persuasive trace that does not match how it actually reached the answer.

License

Inherits the apache-2.0 license from the base model, which ships a LICENSE file.

Related

• Qwen/Qwen3-4B-Thinking-2507 - the base model
• saidutta69/Qwen3-4B-Instruct-2507-heretic - the non-thinking sibling with an enable_thinking toggle
• Qwen3 technical report
• RACER IS OP - Heretic Models - full collection

Abliteration hardware

Recorded in reproduce/reproduce.json:

• GPU: NVIDIA RTX A6000 (44.42 GB VRAM)
• CPU: 11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz
• OS: Linux-6.8.0-85-generic-x86_64-with-glibc2.35
• CUDA / driver: CUDA 13.0, driver 580.178.04
• Software: PyTorch 2.14.1+cu130, Heretic 1.4.0

Heretic notes that KL-divergence and refusal numbers can vary slightly with hardware and library versions.