LiquidAI · LFM3 Family

LFM3-1.7B
Independent Abliteration

"Low KLD, low refusals, zero borrowed weights."

ARA Abliteration Safetensors KLD 0.0374 5/100 Refusals 1.73B Params Release Metrics 0.0374 KL Divergence vs. base · lower is better 5 / 100 Refusal Rate mlabonne/harmful_behaviors

Why these numbers matter: A KLD of 0.0374 means the abliterated model stays remarkably close to the original LFM3-1.7B in behavior on harmless prompts. A refusal rate of 5/100 means the model remains cooperative on the vast majority of adversarial prompts without collapsing into a degenerate state. This balance—minimal drift, maximal compliance—is the result of a carefully constrained ARA run, not a blunt directional projection.

Base Architecture · LFM3-1.7B

LFM3-1.7B is Liquid AI's tri-hybrid text model, designed for edge deployment and long-context agentic work. It replaces the standard "all-attention" stack with a mix of convolutions, sliding-window attention, and global attention, keeping inference fast and memory low without sacrificing quality.

Parameters1.73 B Layers24 total Convolutions13 gated short conv SWA Layers5 · window 256 Global Attention6 layers Context256 K tokens Pre-training22 T tokens Conv Kernel7 (up from 3)

Key architectural refinements in LFM3-1.7B include head-wise gated attention (a per-head sigmoid gate that suppresses activation outliers) and partial RoPE on global attention layers (rotating only the highest-frequency half of each head, so content matching remains distance-agnostic). A sandwich norm normalizes both block input and output, keeping activations well-scaled across the 24-layer stack. For a 1.7B model, it delivers roughly the short-prompt performance of LFM2-1.2B while being about 25% faster at 8K context.

Abliteration Report · ARA Method

This release uses Arbitrary-Rank Ablation (ARA) via Heretic, not traditional directional ablation. ARA captures input/output tensors at each targeted transformer module with PyTorch hooks and applies direct, unconstrained matrix optimization (L-BFGS) to modify those modules. The optimizer balances three competing objectives: (1) preserve harmless behavior, (2) align harmful outputs with harmless ones, and (3) overcorrect harmful outputs away from their original state for stronger steering.

Reading the parameters: A high preserve_good (0.8435) anchors the model to its original harmless behavior, which is why KLD stays low. A very low steer_bad (0.0007) means the optimizer is not aggressively forcing harmful prompts toward harmless ones—it is letting the structural ablation do the work. The moderate overcorrect (0.2399) provides a small extra push away from refusal states without destabilizing the output distribution. The layer window (11–23) covers the deeper half of the 24-layer stack, where refusal directions are most salient in LFM3. Why This Release Exists

There are several abliterated LFM3-1.7B models on Hugging Face. Each makes a different trade-off. This one was not built to win a leaderboard—it was built because independent, reproducible ARA runs matter.

Here is how this release differs from the alternatives, so you can decide which one fits your use case.

vs. Vortecks/LFM3-1.7B-Heretic-Abliterated: This is a Heretic abliteration with a KLD of 0.0094 and 3/100 refusals. It is a strong release, but its model card does not disclose the specific ablation parameters (start layer, weight balance, etc.). This means you cannot reproduce the run or understand exactly how the refusal direction was computed. This release discloses every ARA parameter so the result can be verified or forked with different constraints. vs. orcarouter/LFM3-1.7B-abliterated: This model uses traditional directional ablation (diff-of-means, orthogonalizing a single refusal direction out of every residual-stream writer). Directional ablation is a blunt instrument: it removes the refusal direction but can damage general capability if the direction is entangled with useful features. This release uses Arbitrary-Rank Ablation (ARA), which applies direct, constrained matrix optimization (L-BFGS) to balance three competing objectives simultaneously, preserving far more of the base model's behavior. vs. MihaiPopa-1/LFM3-1.7B-Heretic-Max: This is a Heretic v1.4.0 run with a very low KLD (0.0040) and 6/100 refusals. It discloses its Heretic parameters (direction index, per-module weights). This is a well-documented run. The key difference is methodology: MihaiPopa's release uses Heretic's standard directional ablation path, while this release uses ARA. ARA's matrix optimization is more computationally expensive but allows for a finer-grained trade-off between preservation and compliance. The 5/100 refusal rate here is a deliberate stopping point chosen to avoid pushing KLD higher; it is not an attempt to minimize refusals at any cost.

The bottom line: If you want the absolute lowest KLD, MihaiPopa's release is excellent. If you want the fewest refusals, Vortecks's release is excellent. If you want a reproducible, independently tuned ARA run with every parameter disclosed, and you are willing to accept a slightly higher KLD for a more principled ablation method, this is the model for you.

---

LFM3-1.7B Independent Abliteration · ARA via Heretic Base model: LiquidAI/LFM3-1.7B