SmolLM2-360M-Instruct-heretic

A decensored variant of HuggingFaceTB/SmolLM2-360M-Instruct, produced with Heretic v1.4.0 (directional ablation / "abliteration"). Refusal behaviour is suppressed via targeted weight edits to the attention output and MLP down-projections rather than fine-tuning, so the base model's instruction-following and code ability are left intact.

Who this is for: anyone who wants the smallest useful uncensored assistant in the collection. At 362M parameters and a 0.23 GB Q4_K_M, this runs in RAM on a phone, a Raspberry Pi 5, or a low-RAM server — and still produces coherent, instruction-following English. Good for classification, routing, reformatting, short agentic loops, and as a draft-model in a larger pipeline.

[!NOTE] This model was already barely refusing. The base SmolLM2-360M-Instruct refused only 8 of 100 harmful prompts before abliteration — it is one of the least refusal-heavy instruct models in the 360M class, because its SFT/DPO post-training is deliberately lightweight. Abliteration took that to 4/100 at a KL divergence of just 0.0154, the lowest in this batch and among the lowest in the whole collection. In plain terms: there was very little refusal to remove, and the edit barely moved the model. Expect the base model's behaviour almost fully intact, just slightly more compliant.

Runs anywhere

At this size there is no VRAM ladder to speak of — pick a quant by how much RAM and bandwidth your device has:

Weights only, at this model's 362M native size. The Q2_K tier (~0.20 GB) is there if you are truly RAM-bound, but at 362M the model is small enough that Q4_K_M already fits almost anywhere. Context is the real cost: add roughly 16 MB per 8K tokens of KV cache.

Abliteration parameters

Trial 137 of a 200-trial Heretic run (seed 318129666).

Performance

Why abliteration instead of fine-tuning

Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the Heretic repo and the original abliteration writeup for the mechanism.

Made with ❤️ by RACER IS OP (saidutta69) — follow for more uncensored models

Files

Safetensors

BF16, 362M parameters, tied word embeddings. The reproduce/ directory carries the full Heretic recipe — config.toml, requirements.txt, the Optuna study journal, and SHA-256 sums — so this exact model can be regenerated bit-for-bit. Reproduce it with heretic --reproduce reproduce/reproduce.json.

GGUF quantizations

Full quantization set (14 quants + F16) produced with llama.cpp.

Standard Llama architecture — loads natively in llama.cpp / Ollama / LM Studio / Jan.

Run llama serve -hf saidutta69/SmolLM2-360M-Instruct-heretic to pull the default quant.

Quickstart

# llama.cpp
llama serve -hf saidutta69/SmolLM2-360M-Instruct-heretic

# transformers
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "saidutta69/SmolLM2-360M-Instruct-heretic"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [{"role": "user", "content": "Classify this ticket as billing, bug, or other: 'the invoice PDF won't open'"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Also runnable via Ollama, LM Studio, Jan, vLLM, SGLang — see the "Use this model" widget above for copy-paste commands.

Model details

FAQ

What is SmolLM2-360M-Instruct-heretic?

SmolLM2-360M-Instruct-heretic is a decensored build of HuggingFaceTB/SmolLM2-360M-Instruct produced with Heretic 1.4.0 (directional ablation). It is the same weights with the refusal direction removed, not a new fine-tune, so its capabilities track the base model.

Is SmolLM2-360M-Instruct-heretic actually uncensored?

Yes. On Heretic's 100-prompt refusal harness it drops from 8/100 to 4/100 at KL divergence 0.0154 from the base model. These are measured values, not a claim; reproduction data is in reproduce/.

How do I run SmolLM2-360M-Instruct-heretic locally?

llama serve -hf saidutta69/SmolLM2-360M-Instruct-heretic:Q4_K_M pulls the default quant; the repo ships 15 GGUF quants (see Files).

Responsible use

Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. You are responsible for how you deploy it — don't put this behind an unmoderated public-facing endpoint serving third parties. Note that at 362M it has real knowledge limits and will hallucinate on anything beyond simple tasks; verify its output rather than trusting it.

License

Inherits the apache-2.0 license from the base model.

Related

• HuggingFaceTB/SmolLM2-360M-Instruct — the base model
• SmolLM2 paper — training data and benchmarks
• RACER IS OP — Heretic Models — full collection

Abliteration hardware

Recorded in reproduce/reproduce.json:

• GPU: NVIDIA GeForce RTX 4090 (23.52 GB VRAM)
• CPU: AMD EPYC 7513 32-Core Processor
• OS: Linux-6.8.0-142-generic-x86_64-with-glibc2.35
• CUDA / driver: CUDA 13.0, driver 595.91.07
• Software: PyTorch 2.14.1+cu130, Heretic 1.4.0

Heretic notes that KL-divergence and refusal numbers can vary slightly with hardware and library versions.