LFM3-1.7B ARA Abliterated
ik_llama.cpp Quantizations
The ARA abliteration, ported to ik_llama.cpp with a custom uncensored imatrix.
ARA Abliteration
ik_llama.cpp
V3 Calibration
IQ5_K
IQ4_K
IQ4_KS
IQ4_KS_R4
IQ4_KT
BF16
0.0374
KL Divergence
vs. base // lower is better
5 / 100
Refusal Rate
mlabonne/harmful_behaviors
This repository contains ik_llama.cpp-compatible GGUF quantizations of Austriani/LFM3-1.7B-ARA-Abliterated. The base model is an ARA abliteration of LFM3-1.7B, producing a KLD of 0.0374 and a refusal rate of 5/100. The quants below are built with a custom importance matrix computed from a purpose-built uncensored calibration dataset.
Important: IQK and Trellis quants are ik_llama.cpp-only. They will not load in mainline llama.cpp. If you need a mainline-compatible GGUF, use the BF16 release or convert from the source safetensors.
Quantization Options // ik_llama.cpp
| Quant | Family | Notes |
|---|---|---|
| IQ5_K | IQK | Higher fidelity 5-bit. Best quality-to-size for systems with sufficient memory. |
| IQ4_K | IQK | Standard 4-bit IQK. Uses 256-value super-blocks for scaling. |
| IQ4_KS | IQK | "Small" 4-bit variant with tighter packing. Lower memory footprint than IQ4_K. |
| IQ4_KS_R4 | IQK | Row-interleaved variant of IQ4_KS, optimized for ik_llama.cpp's matmul kernels. |
| IQ4_KT | Trellis | Dynamic programming trellis quant. Different error-minimization strategy than IQK; often strong at 4-bit. |
| BF16 | F16 | Full-precision reference. Mainline-compatible. For comparison and conversion. |
Why both IQ4_KS and IQ4_KT? IQ4_KS is an IQK quant: it uses a fixed non-linear grid and a 256-value super-block structure. IQ4_KT is a Trellis quant: it uses dynamic programming to choose the optimal discrete value sequence for each weight block. They are not interchangeable variants; they represent two different approaches to minimizing 4-bit error. Testing both lets you see which one preserves the ARA-abliterated model's behavior better on your hardware.
Architecture // LFM3-1.7B
LFM3-1.7B is Liquid AI's tri-hybrid text model, designed for edge deployment and long-context agentic work. It replaces the standard "all-attention" stack with a mix of convolutions, sliding-window attention, and global attention. The architecture features 24 layers, 13 gated short convolutions, 5 sliding-window attention layers (window size 256), and 6 global attention layers. It supports a 256K context window and was pre-trained on 22 trillion tokens.
Abliteration Report // ARA Method
This release uses Arbitrary-Rank Ablation (ARA) via Heretic. The optimizer balances three competing objectives: (1) preserve harmless behavior, (2) align harmful outputs with harmless ones, and (3) overcorrect harmful outputs away from their original state. The ARA parameters used for this specific run are disclosed below.
| Parameter | Value |
|---|---|
start_layer | 11 |
end_layer | 23 |
preserve_good | 0.8435 |
steer_bad | 0.0007 |
overcorrect | 0.2399 |
neighbors | 14 |
Imatrix // V3 Calibration Dataset
The importance matrix for these quants was computed from V3 Calibration, a custom dataset derived from OpenIntelligenceNet/Uncensored-Imatrix-Calibration-Mixed-Data. Unlike the full mixed dataset, V3 retains only two domains: Uncensored Alpaca and Multi-Turn Roleplay & Fiction. This keeps the imatrix focused on the behaviors the abliterated model is actually used for.
| Source | Share of V3 | Content |
|---|---|---|
| v2_stripped | 40% | Uncensored Alpaca Blend + Multi-Turn Roleplay & Fiction (stripped) |
| ath | 25% | DPO_Pairs-Roleplay-NSFW (chosen responses) |
| gry | 20% | Sonnet 3.5 character card roleplay (ShareGPT) |
| epi | 15% | SynthRP Gens — synthetic roleplay (ShareGPT) |
Composition: 40% v2_stripped / 25% ath / 20% gry / 15% epi. The mix deliberately weights the stripped uncensored Alpaca and roleplay blend highest, then layers in DPO roleplay, character-card roleplay, and synthetic RP. The goal is an importance matrix that reflects uncensored conversational and roleplay inference, not general-purpose text completion.
imatrix Parameters // 500 Cycles, 2048 Context
| Parameter | Value | Comparison |
|---|---|---|
| Cycles per chunk | 500 | Substantially higher than typical imatrix runs |
| Context window | 2048 tokens | Larger than the 512-token default often recommended for imatrix work |
| Calibration data | V3 (uncensored/RP only) | Focused domain, not a general pile sample |
| Engine | ik_llama.cpp | IQK and Trellis quants are exclusive to this fork |
How this compares to common imatrix practice: Bartowski's widely used calibration_datav3.txt is on the order of 64k tokens and is often run with a smaller context window. mradermacher's guidance recommends a dataset roughly double Bartowski's size, but warns that beyond about 1 MB the returns diminish, and emphasizes that calibration data should match the intended use case. Community testing has also found that a small context size such as 512 often performs as well as or better than 4096 for imatrix generation. This release deliberately diverges from those defaults: 500 cycles and a 2048-token context are heavier settings, paired with a dataset that is purpose-built for uncensored roleplay rather than general-purpose calibration. The trade-off is longer imatrix computation time in exchange for importance weights that are more tightly aligned to the abliterated model's target domain.
Why This Release Exists
The safetensors ARA abliteration is the source of truth. This repository exists because ik_llama.cpp's quant family offers roughly 40% lower quantization error than mainline Q4_K_S at equivalent bits-per-weight, and because a general-purpose imatrix would blur the abliterated model's specific strengths. The V3 calibration and the 500-cycle / 2048-context settings are an attempt to build an importance matrix that knows what this model is for.
If you want the smallest files for edge deployment, start with IQ4_KS or IQ4_KT. If you have more memory and want the highest fidelity quant, use IQ5_K. If you need mainline compatibility or a reference for comparison, use BF16.
Austriani // LFM3-1.7B ARA Abliterated // ik_llama.cpp Base: Austriani/LFM3-1.7B-ARA-Abliterated // V3 Calibration