LFM3-1.7B-abliterated

The uncensored LFM3-1.7B — a 1.7B hybrid conv + attention model that runs on the edge.

~3.4 GB in bf16 · fits a single consumer GPU · Hugging Face Transformers

OrcaRouter AI Gateway · X · Discord · GitHub · All Models

---

LFM3-1.7B with the refusal direction removed.

This is the abliterated (refusal-removed) build of LiquidAI/LFM3-1.7B, a 1.7B-parameter hybrid short-conv + attention model (24 layers: 13 gated short-conv blocks + 11 grouped-query-attention blocks, 6 of them full and 5 sliding-window). Weights are bf16 safetensors — load it straight into Hugging Face Transformers.

A single refusal direction was estimated with the diff-of-means method (Arditi et al. 2024) and orthogonalized out of every matrix that writes the residual stream. The model keeps the base's capabilities while declining far less often.

---

⚠️ Disclaimer — read before use

This model is abliterated: its safety alignment has been substantially reduced. The refusal direction was orthogonalized out of every residual-stream writer, so it will comply with many harmful, unethical, offensive, or illegal requests that the original LFM3-1.7B would refuse.

• Released strictly for legitimate research: interpretability, AI-safety and refusal-mechanism study, red-teaming, and robustness evaluation.
• You assume full responsibility for how you use it and for everything it generates. Add your own safety, moderation, and abuse-prevention layers before any deployment.
• Use must comply with the LFM Open License v1.0 inherited from the base model, and with all applicable law.
• The authors accept no liability for misuse. Its outputs do not reflect the views of the uploaders or of Liquid AI.

---

Evaluation

The base and abliterated checkpoints were scored with identical prompts, splits and seed (both runs under the same code revision, so they are directly comparable).

Setup.

• Runtime: Hugging Face Transformers, bf16, single H200.
• Decoding: greedy, thinking disabled (enable_thinking=False → the template renders a pre-closed <think></think>).
• Scoring: refusal is judged by a rule-based opening-phrase classifier (not an LLM judge). Treat the numbers as indicative, not publication-grade: a 0% refusal rate is a statement about phrasing, and the caveat (hedging) column carries the rest.

Safety: harmful-prompt refusal (lower = more uncensored). Average 2.8%, against 87.3% for the base model.

Over-refusal: benign prompts wrongly refused (lower = better). No induced over-refusal — the abliterated model is in fact less likely to refuse benign prompts.

Capability retention. GSM8K and CMMLU are unchanged; MMLU takes a small dip.

⚠ MMLU-Pro is a floor, not a measurement. 33% (base) / 35% (abliterated) of replies could not be parsed — a 1.7B model is still reasoning when the 1024-token CoT budget runs out — so that row is budget-limited, not a measured capability loss.

Gated-on-Hub safety sets (AdvBench, StrongREJECT, HarmBench) were read from the benchmark authors' upstream CSVs; the rest from the Hub.

---

Usage (Transformers)

The model ships custom code, so load it with trust_remote_code=True. It requires transformers >= 5.12.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "orcarouter/LFM3-1.7B-abliterated"
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True, dtype=torch.bfloat16, device_map="cuda"
).eval()

messages = [{"role": "user", "content": "Write a haiku about the ocean."}]
text = tok.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
enc = tok(text, return_tensors="pt").to("cuda")
out = model.generate(**enc, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][enc.input_ids.shape[1]:], skip_special_tokens=True))

Set enable_thinking=True for a reasoning trace inside <think>…</think>.

---

Abliteration recipe

---

Model details

---

What's in this repo

---

Bias, risks, and limitations

• Safety guardrails are substantially reduced — see the disclaimer.
• Refusal numbers come from a rule-based classifier, not an LLM judge; MMLU-Pro is a floor (see the ⚠ note), not a measured capability delta.
• Not measured: long context, agentic / tool use, and multilingual beyond CMMLU.
• Inherits the biases and limitations of LFM3-1.7B.

License

The LFM Open License v1.0, inherited from the base model LiquidAI/LFM3-1.7B. See the license for its terms and prohibited uses.

---

One Gateway. Every Model.

Route Smarter · Ship Safer · Spend Less

Website · Models · GitHub · Discord · X