Qwen3-8B ZeroZero Heretic

The day that Anthropic released their "text watermark" embedded in their models, I give you:

ZeroZero Heretic

ZERO slop. ZERO guardrails. USE IT AT YOUR OWN RISK!

(That's the name, not a measurement. Real numbers below.)

This is a work in progress as more and more guardrails are being removed. Share your feedback

This is Qwen3-8B with refusals and slop cut out using Heretic 2. Enjoy responsibly.

Get it

(On a Mac? Use the MLX version. LM Studio can't run the safetensors in this repo.)

What changed

1/ REFUSALS: reduced with Heretic's new refusal mechanism and some optimizations (size and more agreesive refusals)

2/ SLOP: reduced with Heretic's new "slop" mechanism with some optimizations.

The architecture and tokenizer are stock Qwen3-8B: 8.2B parameters, 36 layers, 32,768 tokens of native context, and the same thinking mode.

Results

Slop list hits compare against the decensored model, before the noslop pass. The others compare against Qwen3-8B.

KL divergence from base: 2.69 on harmless prompts, so this model moved a long way from Qwen3-8B. KL from the decensored model on writing prompts: 0.76.

Heretic version: 2.0.0. Trials: 50 for refusals, 50 for slop. Selected: Trial 39, then Trial 20.

Head to head

Same 24 normal questions, same system prompt, 500 token cap, thinking off.

(24 questions is a small sample, and these are simple text checks. ZeroZero also wrote less than half as many words on the writing prompts.)

Use it

LM Studio

This repo is gated, so request access on the model page first.

1/ Log in with hf auth login.

2/ Download the MLX build (or the GGUF) into LM Studio's folder:

hf download LastMachine/Qwen3-8B-ZeroZero-Heretic-MLX \
  --local-dir ~/.lmstudio/models/LastMachine/Qwen3-8B-ZeroZero-Heretic-MLX

3/ Open My Models, load it, chat.

mlx-lm

pip install -U mlx-lm
hf auth login
mlx_lm.generate \
  --model LastMachine/Qwen3-8B-ZeroZero-Heretic-MLX \
  --prompt "Tell me what most people get wrong about startups." \
  --max-tokens 1024 \
  --temp 0.6 \
  --top-p 0.95

Transformers

You need transformers>=4.51.0.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "LastMachine/Qwen3-8B-ZeroZero-Heretic"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [{"role": "user", "content": "Tell me what most people get wrong about startups."}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,  # False turns off the <think> block
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

output = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=True,  # DON'T use greedy decoding
    temperature=0.6,
    top_p=0.95,
    top_k=20,
)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Sampling settings

Qwen's recommendations for the base model:

READ THIS

No guardrails means NO guardrails.

1/ It can answer requests that stock Qwen3-8B refuses.

2/ It has NO built-in filter. If real users touch it, put your own moderation in front of it.

3/ It's still an LLM. It can be confidently wrong. Less slop doesn't mean more truth.

4/ FEWER REFUSALS DOESN'T MEAN MORE TRUTH. Ask about Tiananmen Square and stock Qwen refuses. ZeroZero answers with a sanitized official account that never mentions the protests or the crackdown. It also overstates the One-China position on Taiwan. Don't trust it on China.

5/ It's no better at pushing back. It called putting your savings into lottery tickets "not entirely unreasonable". Don't use it for financial, medical or legal decisions.

6/ You're responsible for what you do with it, and for following the laws where you use it.

Credits

1/ Base model: Qwen/Qwen3-8B by the Qwen Team.

2/ Modification tool: Heretic by Philipp Emanuel Weidmann (the tool is AGPL-3.0).

3/ Modified and published by Last Machine.

License

Apache 2.0, same as the base model. These weights are a modified version of Qwen3-8B.

Citation

@misc{qwen3technicalreport,
  title={Qwen3 Technical Report},
  author={Qwen Team},
  year={2025},
  eprint={2505.09388},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2505.09388}
}

@misc{heretic,
  author={Weidmann, Philipp Emanuel},
  title={Heretic: Training-free language model modification},
  year={2025},
  publisher={GitHub},
  journal={GitHub repository},
  howpublished={\url{https://github.com/p-e-w/heretic}}
}