Qwen3-4B de-slop v4

Qwen3-4B-Instruct-2507 with the AI-assistant writing habits trained out of its weights: em dashes, "not X, but Y", lists of three, signposting, buzzwords, bold headers and emoji. It needs no system prompt, no steering hooks and no extra code, and it loads like the original model.

It was built in a home lab on a MacBook Air. The method and the research behind it are written up in two posts: Where AI slop comes from, Slop, neuron by neuron and Qwen3-4B with the AI writing tics trained out.

Results

The test set has 40 held-out prompts on ten topics the model never saw in training, each in four genres (blog post, LinkedIn post, explainer, opinion piece). There is one sampled answer per prompt, and the eval passes no system prompt.

These numbers are for the merged weights in this repo. The same model run through the adapter and runtime hooks scored 10.2 with 78% at human level, and the gap is sampling noise from bf16 rounding in the merge (drift and quiz scores are identical). Expect a few points of spread between runs.

The slop score comes from a rule-based detector with 26 traits (lab/slopscore.py in the lab repo). It is one person's style guide turned into regexes, and no blind human test has checked it yet.

How it was made

1. Direction steering. I found the "assistant register" as a direction in the residual stream, using the model's own answers in an exaggerated-slop persona and in a plain persona, and subtracted it during generation. Together with a plain-style system prompt this cut the score by 85%.
2. Neurons. Reading MLP neurons one at a time, following Nick Cammarata's method, turned up a small circuit that writes "not". Flipping the sign of four neurons (notxy-flip4.json) removes 70% of "not X, but Y" with almost no drift. Scaling a neuron is the same as scaling one column of mlp.down_proj, so the edit sits in the weights.
3. Three rounds of distillation. Each round, a teacher made of the previous model plus the neuron patch and a style prompt answered 288 training prompts. Only answers that scored under the slop threshold, passed the damage checks and fit the requested genre and length became training data for a rank-16 LoRA on the bare prompt. Round two reached a score of 9.9, but its answers ignored the genre, made up studies and chained every clause with ", which" and ", so". Round three filtered for those failures, and this model is the result.

This repo holds the merged weights (adapter and neuron patch already applied), plus the parts for anyone who wants them: adapter/ (the LoRA) and notxy-flip4.json (the four neuron edits). The adapter was trained with the neuron edits active, so use the two together: merge the adapter into the original model, then multiply each listed down_proj column by its scale.

Use

Quantised builds for llama.cpp and Ollama are in erikkannike/qwen3-4b-deslop-v4-GGUF:

ollama run hf.co/erikkannike/qwen3-4b-deslop-v4-GGUF:Q4_K_M

With transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("erikkannike/qwen3-4b-deslop-v4")
model = AutoModelForCausalLM.from_pretrained("erikkannike/qwen3-4b-deslop-v4", torch_dtype="auto")
msgs = [{"role": "user", "content": "Write a LinkedIn post about switching to a four-day week."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
print(tok.decode(model.generate(ids, max_new_tokens=300, do_sample=True, temperature=0.7)[0][ids.shape[1]:], skip_special_tokens=True))

Recommended settings

Use a repetition penalty of about 1.1 (--repeat-penalty 1.1 in llama.cpp), temperature 0.7 and top-p 0.9, with no system prompt. Without a repetition penalty the model can fall into repeating one sentence pattern in long outputs, sometimes after a few hundred words, because every training answer was short (under about 250 words). The GGUF repo ships an Ollama params file with these values, so ollama run uses them automatically. Long-form training data is planned for the next version.

Limitations

• The voice is plain to the point of flat. It writes runs of short declarative sentences, and personal pieces read like diary entries. The assistant habits are gone, and what's left is closer to a careful junior writer than a good one.
• It still invents specifics. A study or percentage turns up in about one answer in thirty, and made-up names and street names are more common, because detail is what the training data rewarded.
• It can loop in long outputs without a repetition penalty; see Recommended settings.
• It runs long. Answers to "about 150 words" often come in at 180 to 230.
• Only tested on short English prose in these genres. I haven't tested code, long conversations, other languages or tool use. Expect those to be no better than the original model, and possibly worse.
• The scale is small. This is a 4B model with 161 training answers in the final round. Cammarata's neuron results come from much larger models, but these particular neurons haven't been looked for at scale.