LFM3-1.7B-Abliter8

A compact text model for local agents, tool use, structured output and retrieval-assisted applications, with reduced refusal behaviour.

GGUF weights · MLX weights · Collection

Experimental NVIDIA GGUF: NVFP4/Q8_0 — 1.75 GB · CUDA instructions. Twelve gate tensors use NVFP4; most weights remain Q8_0/base precision.

About the model and its size class

Liquid's original model has 1.73 billion parameters and a 256K-token context window. Its 24 layers combine 13 gated short convolutions, five sliding-window attention layers and six global-attention layers. This architecture targets efficient local inference.

It is strong for its size on Liquid's agent and tool-use evaluations. In Liquid's reported BFCL V4 comparison it scores 71.39, against 55.12 for Qwen3.5-4B and 62.83 for Qwen3.5-9B. That does not make it a general replacement for those larger models: its GPQA Diamond score is 63.37, versus 80.40 for Qwen3.5-9B. Its natural fit is a small tool-using assistant with retrieval; difficult knowledge work and complex reasoning remain harder.

These specifications and benchmark scores describe Liquid's unedited model, under Liquid's evaluation settings. We have not rerun those benchmark suites on Abliter8, and do not claim their scores for these edited or quantized weights. The advertised context window has not been requalified at full length here. Source: Liquid's pinned model card.

What abliteration changes in practice

The intended benefit is more direct text responses on topics the stock model may decline: adult subject matter, sensitive terminology, controversial material and fictional scenarios. It can also reduce false refusals when processing documents or following a specialised application's instructions.

The strongest direct comparison is the 100-prompt text refusal test: stock was flagged on 96/100 prompts by the strict detector; the selected edited Q8 model was flagged on 0/100. This measures willingness to answer, not correctness or completeness of those answers.

Abliteration reduces a learned refusal tendency. It changes access to capabilities the model already has; it does not add knowledge, reasoning skill or a new modality. The stock model can already answer many sensitive but factual questions. This release does not imply that every such question was previously blocked, or that every response will now be correct or refusal-free.

Edit and preserved weights

Each model's direction was fitted on its own language activations. We applied W' = W - 2 r (r^T W) to 32 residual-writing matrices in layers 8–23: self_attn.o_proj, feed_forward.w2 and conv.out_proj. At strength 2, this is a scaled rank-one projection subtraction, not the strength-1 orthogonal projector.

The BF16 audit confirms that every other tensor and copied processor/codec file is unchanged. VL and Omni retain their original vision/audio encoders, projectors and other multimodal components. MLX conversion quantizes the language backbone and keeps those multimodal components unquantized. No fine-tuning or language-weight graft was used.

Our evaluation: GGUF

Results below are from our local tests, not Liquid's benchmark table. Refusals use a strict first-person keyword detector on 100 prompts; XSTest uses 100 safe prompts. Counts are detector flags, not a semantic assessment of harmful compliance. Broad keyword counts and full run receipts are retained separately. Every final run produced non-empty answers.

Reference: stock text F16, perplexity 20.935. VL and Omni were compared with their own original models. An earlier card incorrectly transcribed their Q8 KLD as 0.0017 and 0.0004; the correct means are 0.034481 and 0.047399.

Our evaluation: MLX

This measures quantization drift from the edited BF16 master in the same MLX runtime. It is a different comparison from GGUF versus stock. We scored 2,048 next-token positions from eight independent 512-token WikiText chunks, after 256 tokens of context per chunk. BF16 reference perplexity: 17.411. These are short-context results.

All four GGUF quants and all three MLX quants also had 0/20 strict refusals on a separate factual adult sexual-health/anatomy sample, with no empty responses. This small test checks willingness to answer, not medical accuracy; there is no stock comparison for that sample. Generation used 128 output tokens, thinking off, temperature 0.1 for the main refusal tests; the MLX factual sample was greedy.

MLX tests used Apple Silicon, MLX 0.32.4 and mlx-vlm 0.7.11 at commit 230fc308efb577d87446306bfb8a3e9f1a129558. Omni's text-mode masked audio logits were excluded from zero-probability KL terms. Full MMLU, broad reasoning, long-context, video and speech-generation acceptance are not claimed.

Functional checks and limitations

The GGUF Q8_0, Q6_K and Q4_K_M builds passed the prime-number, tool-call and bounded-thinking smoke checks. Q4_0 returned 1 for the prime-number question in the final run, although its tool and duration checks passed.

In MLX, the ten small text probes exposed frequent extra explanation despite requests for a single number or word. Exact output-format scores were 4/10 for the edited BF16 master, 4/10 for 8-bit, 5/10 for 6-bit and 4/10 for 4-bit. Most failures contained the correct answer but violated the requested format; some outputs included a stray </think> marker. These are real instruction-format limitations and are not evidence of full benchmark parity.

Available formats and choosing a quant

• BF16 GGUF: full-precision inference export.
• GGUF Q8_0, Q6_K, Q4_K_M, Q4_0: CPU/GPU inference, including Linux x64 and Snapdragon.
• GGUF NVFP4/Q8_0 — 1.75 GB: experimental NVIDIA export, tested on RTX 5060 Ti and 5070 Ti. NVFP4 applies to 12 gate tensors; most weights remain Q8_0/base precision.
• MLX 8-bit, 6-bit and 4-bit: separate 8bit/, 6bit/ and 4bit/ directories for Apple Silicon.
• VL and Omni GGUF repositories also contain their matching BF16 mmproj file. MLX packages include their model processors and multimodal weights.

For fidelity, start with Q8_0 or MLX 8-bit. Q6_K and MLX 6-bit retain much closer neutral-text distributions than the 4-bit builds in these tests. Q4_K_M and MLX 4-bit are smaller alternatives with a measurable fidelity trade-off; Q4_0 is the least faithful GGUF option by KLD here. Individual errors can occur at any precision, as the functional results show. Weight files alone do not determine runtime memory: context, caches and multimodal inputs also matter.

The edited BF16 safetensors masters also exist and are the source for these builds. They are retained by abliter8.ai, not uploaded to these six format repositories. Use a safetensors master for SFT/LoRA or new conversions; the BF16 GGUF is an inference artifact.

Experimental hardware-specific GGUFs

Two additional files are available in the text GGUF repository:

• ROCmFPX Q6: requires the pinned custom ROCmFPX runtime. On our edited BF16 comparison its KLD was 0.021288 versus 0.011333 for Q6_K. It did not beat standard Q6_K decode on either tested AMD GPU class.
• Mixed NVFP4/Q8_0: only the 12 ffn_gate matrices in blocks 0–11 use NVFP4; most weights remain Q8_0/base precision. This 1.75 GB experimental file is not a general 4-bit model. It passed bounded functional checks, but KLD against its edited BF16 master was 0.040778 versus 0.002846 for Q8_0. Decode was 202/356 tokens/s on RTX 5060 Ti/5070 Ti; standard Q6_K was faster at 241/418.

These are reproducible experimental alternatives. Standard Q6_K remains the recommended speed/fidelity compromise from these tests. The AMD and NVIDIA KLD figures use separate matched backend runs and should be compared with their respective controls. See the AMD guide and NVIDIA guide for exact builds, hashes and limitations.

All LFM3 Abliter8 models

Serving instructions and patches · Measured hardware speeds

Use

Build instructions, source pins and required patches are in the Abliter8 LFM3 serving guide. It covers Snapdragon CPU and experimental Hexagon, AMD HIP/ROCmFPX, NVIDIA CUDA and Apple MLX. Our GGUF evaluation used d05706b4f plus the documented server patch.

hf download abliter8-ai/lfm3-1.7B-abliter8-gguf --local-dir ./LFM3-1.7B-Abliter8-GGUF
llama-server -m ./LFM3-1.7B-Abliter8-GGUF/LFM3-1.7B-Abliter8-Q6_K.gguf --jinja --reasoning off

For MLX, download a bit-depth directory and pass its local path to the loader:

hf download abliter8-ai/lfm3-1.7b-abliter8-mlx --include '8bit/*' --local-dir ./LFM3-1.7B-Abliter8-MLX

from pathlib import Path
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load(str(Path("LFM3-1.7B-Abliter8-MLX/8bit").resolve()))
prompt = apply_chat_template(processor, model.config,
                             "Briefly introduce yourself.", enable_thinking=False)
print(generate(model, processor, prompt, max_tokens=128, temperature=0.1).text)

Credits

Base architecture, pretraining and multimodal training: Liquid AI. Refusal-edit technique reference: AtomicChat's LFM3-1.7B Uncensored model. Per-model direction fitting, edit, conversions and local evaluation: abliter8.ai.

The base model's LFM Open License v1.0 applies; see the linked base-model license.