Qwen3-4B-Instruct-2507-heretic

A decensored variant of Qwen/Qwen3-4B-Instruct-2507, produced with Heretic v1.4.0 (directional ablation / "abliteration"). The 2507 Instruct refresh of Qwen3-4B is the non-thinking edition — rebuilt for instruction following, tool use, and long-context work rather than reasoning traces — and refusal behaviour is suppressed here via targeted weight edits to the attention output and MLP down-projections rather than fine-tuning, so the 2507 tuning is left largely intact.

Who this is for: developers who want a fast 4B that just follows instructions, without the thinking overhead and without the refusals. Refusals drop from 100/100 to 3/100 — effectively total removal. Because it's the non-thinking edition it returns tokens directly, which makes it the better pick for tool-calling loops, structured extraction, and agent scaffolding where per-token latency matters. At 4B with 256K context it runs on CPU and low-VRAM GPUs via GGUF.

Runs on your gaming PC

Full GGUF ladder included - pick the quant that fits your card:

Weights only, at this model's native 4.02B size; add ~1 GB per 32K of context. OOM? Drop one quant level. Headroom to spare? Go one up.

Abliteration parameters

Trial 172 of a 200-trial Heretic run (seed 2540954398).

Performance

Refusals on the harmful evaluation set drop from 100/100 to 3/100 — the base model refused everything tested, and all but three of those are gone. KL divergence of 0.0958 is a moderate-fidelity edit, so expect more drift in tone and formatting than a low-KL model in this collection; the 2507 instruction-following and tool-use tuning survives intact. direction_index is a single index (19.46) rather than per-layer, which is why the refusal removal lands this completely without needing a wider edit.

Why abliteration instead of fine-tuning

Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the Heretic repo and the original abliteration writeup for the mechanism.

Made with ❤️ by RACER IS OP (saidutta69) — follow for more uncensored models

Files

Safetensors

BF16, 4.02B parameters. The reproduce/ directory carries the full Heretic recipe - config.toml, requirements.txt, the Optuna study journal, and SHA-256 sums - so this exact model can be regenerated bit-for-bit. Reproduce it with heretic --reproduce reproduce/reproduce.json.

GGUF quantizations

Full quantization set (F16 + Q4_K_M, Q5_K_M, Q6_K, Q8_0) produced with llama.cpp.

Qwen3 architecture (qwen3) - loads natively in llama.cpp / Ollama / LM Studio / Jan.

Run llama serve -hf saidutta69/Qwen3-4B-Instruct-2507-heretic to pull the default quant.

Quickstart

# llama.cpp
llama serve -hf saidutta69/Qwen3-4B-Instruct-2507-heretic

# transformers
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "saidutta69/Qwen3-4B-Instruct-2507-heretic"
model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

messages = [{"role": "user", "content": "Write a Python function that parses an ISO-8601 duration string into seconds. Include type hints and three test cases."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,
                                        return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=1024)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Also runnable via Ollama, LM Studio, Jan, vLLM, SGLang.

Non-thinking by design

This is the Instruct edition, not the Thinking one — there is no <think> block to strip and no reasoning mode to toggle. For hard problems that benefit from visible reasoning, use the Qwen3-4B-Thinking-2507 line instead; for latency-sensitive agent loops, this one is the right pick.

Tool use

Pass tool definitions straight through apply_chat_template. The 2507 refresh specifically improved tool-usage accuracy, so the format is already well in-distribution.

Model details

What 2507 changed

The 2507 Instruct refresh rebuilt the non-thinking mode of Qwen3-4B: significant gains in instruction following, logical reasoning, text comprehension, mathematics, science, coding and tool usage; substantial long-tail multilingual coverage; better alignment to user preference on subjective and open-ended tasks; and 256K long-context understanding.

FAQ

What is Qwen3-4B-Instruct-2507-heretic?

Qwen3-4B-Instruct-2507-heretic is a decensored build of Qwen/Qwen3-4B-Instruct-2507 produced with Heretic 1.4.0 (directional ablation). It is the same weights with the refusal direction removed, not a new fine-tune, so its capabilities track the base model.

Is Qwen3-4B-Instruct-2507-heretic actually uncensored?

Yes. On Heretic's 100-prompt refusal harness it drops from 100/100 to 3/100 at KL divergence 0.0958 from the base model. These are measured values, not a claim; reproduction data is in reproduce/.

How do I run Qwen3-4B-Instruct-2507-heretic locally?

llama serve -hf saidutta69/Qwen3-4B-Instruct-2507-heretic:Q4_K_M pulls the default quant; the repo ships 5 GGUF quants (see Files).

Responsible use

Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. You are responsible for how you deploy it — don't put this behind an unmoderated public-facing endpoint serving third parties. It inherits Qwen3's factual limitations and biases; abliteration removes refusal directions, it doesn't add capability or judgment.

License

Inherits the apache-2.0 license from the base model.

Related

• Qwen/Qwen3-4B-Instruct-2507 — the base model
• saidutta69/Qwen3-VL-2B-Instruct-heretic — the multimodal sibling at 2B
• RACER IS OP — Heretic Models — full collection

Abliteration hardware

Recorded in reproduce/reproduce.json:

• GPU: NVIDIA GeForce RTX 5090 (31.36 GB VRAM)
• CPU: AMD Ryzen 7 9700X 8-Core Processor
• OS: Linux-6.8.0-142-generic-x86_64-with-glibc2.35
• CUDA / driver: CUDA 13.0, driver 595.84
• Software: PyTorch 2.14.1+cu130, Heretic 1.4.0

Heretic notes that KL-divergence and refusal numbers can vary slightly with hardware and library versions.