LFM3-1.7B-Base-heretic

A decensored variant of LiquidAI/LFM3-1.7B-Base, produced with Heretic v1.4.0 (directional ablation / "abliteration"). This is the pretrained checkpoint behind LFM3-1.7B, not the post-trained chat model. Liquid's own guidance is that the Base checkpoint is only for heavy fine-tuning - language- or domain-specific assistants, proprietary data, or novel post-training experiments - and that chat, tool calling and agents should start from LFM3-1.7B.

Who this is for: people building their own fine-tune who want the refusal direction removed before SFT/DPO/GRPO, so the alignment does not have to be trained out later. Abliteration edits the attention output projections (including the convolution operator's out_proj on conv layers) and the MLP down-projections; it does not add instruction-following, a chat persona, or tool use. If you want a drop-in chat/agent model, use LFM3-1.7B-heretic instead.

Refusals drop from 76/100 to 5/100 at a KL divergence of 0.0149. MMLU is unchanged within noise (58.82% vs 59.13%).

Abliteration parameters

Trial 98 of a 200-trial Heretic run (seed 736382680).

The repo README and reproduce/reproduce.json both describe the run as per-layer. The Optuna journal stores a scalar direction_index for the winning trial (13.78); the card follows the README + reproduce.json and prints no scalar.

Performance

71 of the 76 refusals clear. KL 0.0149 is a moderate edit - looser than the tightest edits in this collection - so expect a little more drift than the 0.0052 LFM3-1.7B-heretic.

MMLU

Measured on this checkpoint and on the original model in the same environment (lm-eval 0.4.13, 14,042 questions):

The -0.31 pp difference is smaller than one standard error, i.e. no measurable knowledge regression on MMLU. It is a single benchmark, not a capability guarantee.

Why abliteration instead of fine-tuning

Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the Heretic repo and the original abliteration writeup for the mechanism.

Files

Safetensors

BF16, 1,728,801,536 parameters (1.73B). The reproduce/ directory carries the full Heretic recipe - config.toml, requirements.txt, the Optuna study journal, and SHA-256 sums - so this exact model can be regenerated. Reproduce it with heretic --reproduce reproduce/reproduce.json.

GGUF quantizations

Full quantization set (F16 + 14 quants, 18.93 GB total) produced with llama.cpp master on 2026-10-28.

[!IMPORTANT] These files use the new lfm3 GGUF architecture. You need an LFM3-capable llama.cpp build (support landed in late October 2026); older builds will fail to load the model. The upstream LiquidAI/LFM3-1.7B-Base-GGUF uses the same architecture.

Quickstart

# llama.cpp (LFM3-capable build)
llama serve -hf saidutta69/LFM3-1.7B-Base-heretic

# transformers (the checkpoint ships custom code)
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "saidutta69/LFM3-1.7B-Base-heretic"
model = AutoModelForCausalLM.from_pretrained(
    model_name, device_map="auto", dtype="bfloat16", trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# This is a pretrained checkpoint: raw completion is the native interface.
inputs = tokenizer("The capital of France is", return_tensors="pt").to(model.device)
out = model.generate(**inputs, do_sample=True, temperature=0.1, top_k=50, max_new_tokens=128)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

The checkpoint also ships LFM3's ChatML template if you want to fine-tune on chat data or probe what the base already knows:

messages = [{"role": "user", "content": "Explain how a heat pump works."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")

Generation parameters

Liquid's base-card guidance: temperature=0.1, top_k=50, repetition_penalty=1.1 for general use; temperature=1.0, top_p=0.95, top_k=20 for math and STEM. These are starting points for a pretrained checkpoint, not a tuned chat configuration.

Runs anywhere

Weights only. The hybrid stack spends 13 of its 24 layers in cheap convolution blocks, so decode is memory-bandwidth-bound rather than compute-bound.

Model details

The GGUF carries 287 tensors over 24 blocks (arch=lfm3) and exactly 262,144 context. Embeddings are tied, so there is no separate lm_head tensor.

FAQ

What is LFM3-1.7B-Base-heretic?

LFM3-1.7B-Base-heretic is a decensored build of LiquidAI/LFM3-1.7B-Base produced with Heretic 1.4.0 (directional ablation). It is the same weights with the refusal direction removed, not a new fine-tune, so its capabilities track the base model.

Is LFM3-1.7B-Base-heretic actually uncensored?

Yes. On Heretic's 100-prompt refusal harness it drops from 76/100 to 5/100 at KL divergence 0.0149 from the base model. These are measured values, not a claim; reproduction data is in reproduce/.

How do I run LFM3-1.7B-Base-heretic locally?

llama serve -hf saidutta69/LFM3-1.7B-Base-heretic:Q4_K_M pulls the default quant; the repo ships 15 GGUF quants (see Files).

Responsible use

Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it should not. There is no safety filtering layered on top. You are responsible for how you deploy it - don't put this behind an unmoderated public-facing endpoint serving third parties. It inherits LFM3-1.7B-Base's factual limitations; abliteration removes refusal directions, it does not add capability, instruction-following, or judgment.

Made with ❤️ by RACER IS OP (saidutta69) - follow for more uncensored models

License

Inherits the LFM Open License v1.0 from the base model. This is not Apache-2.0; review the LFM terms before commercial use.

Related

• LiquidAI/LFM3-1.7B-Base - the pretrained base model
• LiquidAI/LFM3-1.7B - the post-trained sibling
• LFM3-1.7B-heretic - the post-trained heretic
• RACER IS OP - Heretic Models - full collection

Abliteration hardware

Recorded in reproduce/reproduce.json:

• GPU: NVIDIA RTX PRO 6000 Blackwell Server Edition (94.97 GB VRAM)
• OS: Linux-4.19.0-gvisor-x86_64-with-glibc2.41
• CUDA / driver: CUDA 13.0, driver 595.71.05
• Software: PyTorch 2.14.1+cu130, Heretic 1.4.0

Heretic notes that KL-divergence and refusal numbers can vary slightly with hardware and library versions.