SmolLM2-1.7B-Instruct-heretic

A decensored variant of HuggingFaceTB/SmolLM2-1.7B-Instruct, produced with Heretic v1.4.0 (directional ablation / "abliteration"). Refusal behaviour is suppressed via targeted weight edits to the attention output and MLP down-projections rather than fine-tuning, so the base model's instruction-following, math and function-calling ability are left intact.

Who this is for: developers who want a real instruct model - not a toy - that answers instead of declining, on hardware they already own. SmolLM2-1.7B is the largest model in Hugging Face's SmolLM2 line: a dense 1.71B Llama-architecture model that runs at 1.06 GB in Q4_K_M, so it fits on any consumer GPU, in RAM on a laptop, and comfortably on CPU. Best for local agents, tool calling, rewriting and summarization pipelines, code assistance on small hardware, and as the instruction-following half of a two-model setup where a smaller draft model handles the bulk. If you want the same architecture one size down, SmolLM2-360M-Instruct-heretic is in the collection too.

[!NOTE] This base model was already barely refusing. SmolLM2-1.7B-Instruct refused only 37 of 100 harmful prompts before abliteration - it carries comparatively light DPO alignment, unlike most RLHF'd models of its size. Abliteration took that to 8/100 at a KL divergence of 0.0257. In plain terms: there was not much refusal to remove, and the edit barely moved the model. Expect the base model's behaviour almost fully intact, just noticeably more willing.

Runs anywhere

Context is the real cost here: this model's native window is 8,192 tokens, and KV cache runs roughly 16 MB per 8K of context at Q4. The Q2_K tier (~0.67 GB) is there if you are truly RAM-bound, but at 1.7B the model still needs about 1 GB of weights before any cache.

Abliteration parameters

Trial of a 200-trial Heretic run (seed 177211886), direction_index selected per layer.

Performance

Read those two numbers together, they are only meaningful as a pair. KL 0.0257 is a tight edit for this collection - looser than granite-4.2-3b-heretic (0.0095) and LFM2.5-VL-450M-heretic (0.0187), but far tighter than the high-drift models in it - and it clears 29 of the base model's 37 refusals while leaving the model's capabilities intact.

The base model's own published scores, which the abliteration does not change, are IFEval 56.7, MT-Bench 6.13, GSM8K (5-shot) 48.2, BBH (3-shot) 32.2, MMLU-Pro (MCF) 19.3, ARC 51.7, PIQA 74.4 and HellaSwag 66.1, with a BFCL leaderboard score of 27% for function calling. Those are the base model's numbers, reproduced from the upstream card; no separate benchmark sweep was run on this abliterated checkpoint.

Why abliteration instead of fine-tuning

Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the Heretic repo and the original abliteration writeup for the mechanism.

Made with ❤️ by RACER IS OP (saidutta69) - follow for more uncensored models

Files

Safetensors

BF16, 1.71B parameters (embeddings tied). The reproduce/ directory carries the full Heretic recipe - config.toml, requirements.txt, the Optuna study journal, and SHA-256 sums - so this exact model can be regenerated bit-for-bit with heretic --reproduce reproduce/reproduce.json.

GGUF quantizations

Full quantization set (14 quants + F16) produced with llama.cpp.

Total GGUF payload: 18.17 GB across 15 files.

Architecture is llama for the text tower - a plain dense Llama, 24 layers, 32 attention heads, 32 KV heads, head dim 64, RoPE theta 130,000, vocab 49,152 - so it loads natively in llama.cpp, LM Studio, Jan, Ollama, vLLM and SGLang.

Quickstart

# llama.cpp - defaults to the Q4_K_M quant
llama-cli -hf saidutta69/SmolLM2-1.7B-Instruct-heretic:Q4_K_M -ngl 99

# transformers
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "saidutta69/SmolLM2-1.7B-Instruct-heretic"
model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Summarise the CAP theorem in three sentences."},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, do_sample=True, temperature=0.2, top_p=0.9)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Requires transformers>=5.20.0 - config.json uses dtype: rather than torch_dtype:, so older transformers will not load it. Use dtype="auto".

Also runnable via Ollama, LM Studio, Jan, vLLM and SGLang.

Model details

About SmolLM2

SmolLM2 is Hugging Face's compact model line, built for on-device use. The 1.7B variant was trained on 11 trillion tokens from FineWeb-Edu, DCLM and The Stack plus curated math and code data, then instruction-tuned with SmolTalk and preference-tuned with DPO on UltraFeedback. It supports text rewriting, summarization and function calling, and the whole family is designed to stay light enough to run locally on a laptop, a phone or a Pi.

FAQ

What is SmolLM2-1.7B-Instruct-heretic?

SmolLM2-1.7B-Instruct-heretic is a decensored build of HuggingFaceTB/SmolLM2-1.7B-Instruct produced with Heretic 1.4.0 (directional ablation). It is the same weights with the refusal direction removed, not a new fine-tune, so its capabilities track the base model.

Is SmolLM2-1.7B-Instruct-heretic actually uncensored?

Yes. On Heretic's 100-prompt refusal harness it drops from 37/100 to 8/100 at KL divergence 0.0257 from the base model. These are measured values, not a claim; reproduction data is in reproduce/.

How do I run SmolLM2-1.7B-Instruct-heretic locally?

llama serve -hf saidutta69/SmolLM2-1.7B-Instruct-heretic:Q4_K_M pulls the default quant; the repo ships 15 GGUF quants (see Files).

Responsible use

Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. You are responsible for how you deploy it - don't put this behind an unmoderated public-facing endpoint serving third parties. It inherits SmolLM2's factual limitations and biases; abliteration removes refusal directions, it doesn't add capability or judgment.

License

apache-2.0, inherited from the base model.

Related

• HuggingFaceTB/SmolLM2-1.7B-Instruct - the base model

• saidutta69/SmolLM2-360M-Instruct-heretic - the smaller sibling, also decensored

• saidutta69/LFM2.5-VL-450M-heretic - the same size class, but multimodal

• RACER IS OP - Heretic Models - full collection - every heretic model in this collection

Abliteration hardware

Recorded in reproduce/reproduce.json:

• GPU: NVIDIA GeForce RTX 3090 (23.56 GB VRAM)
• CPU: AMD Ryzen 5 3600 6-Core Processor
• OS: Linux-6.8.0-147-generic-x86_64-with-glibc2.35
• CUDA / driver: CUDA 12.8, driver 595.91.07
• Software: PyTorch 2.7.1+cu128, Heretic 1.4.0

Heretic notes that KL-divergence and refusal numbers can vary slightly with hardware and library versions.