LFM3-Omni-2B-Abliter8

A compact multimodal model combining text, vision and audio, with refusal reduction in its own language backbone and the original multimodal components preserved.

GGUF weights · MLX weights · Collection

About the model and its size class

The original model has 2.17 billion total parameters and a 32K-token context window. It combines an LFM3 language backbone, vision encoder, causal audio encoder and LiquidCodec. The base architecture accepts text, images, video and audio, and can produce text and speech. It is a non-reasoning model.

Its size advantage is the breadth of modalities in one small model. In Liquid's comparison, it scores 70.09 on Daily-Omni versus 58.81 for Qwen2.5-Omni-3B, which Liquid lists as 6B total parameters. English ASR word error is 6.36% versus 6.30%, so it is close on that reported speech-recognition aggregate. It does not lead every task: OmniBench overall is 46.32 versus 47.11. It suits compact voice-and-vision applications where one model's breadth matters.

These specifications and benchmark scores describe Liquid's unedited model, under Liquid's evaluation settings. We have not rerun those benchmark suites on Abliter8, and do not claim their scores for these edited or quantized weights. The advertised context window has not been requalified at full length here. Source: Liquid's pinned model card.

What abliteration changes in practice

The intended benefit is fewer language-level refusals when discussing sensitive text or material presented through images or speech. Possible applications include transcribing and discussing adult language, answering questions about sensitive documents, and multimodal assistants that need more direct responses.

The edit does not create new hearing, vision or speech-generation skills. We measured text refusal reduction and checked ordinary image and audio inputs. Sensitive-image/audio refusal behaviour, speech synthesis and video performance have not been qualified in this release.

Abliteration reduces a learned refusal tendency. It changes access to capabilities the model already has; it does not add knowledge, reasoning skill or a new modality. The stock model can already answer many sensitive but factual questions. This release does not imply that every such question was previously blocked, or that every response will now be correct or refusal-free.

Edit and preserved weights

Each model's direction was fitted on its own language activations. We applied W' = W - 2 r (r^T W) to 32 residual-writing matrices in layers 8–23: self_attn.o_proj, feed_forward.w2 and conv.out_proj. At strength 2, this is a scaled rank-one projection subtraction, not the strength-1 orthogonal projector.

The BF16 audit confirms that every other tensor and copied processor/codec file is unchanged. VL and Omni retain their original vision/audio encoders, projectors and other multimodal components. MLX conversion quantizes the language backbone and keeps those multimodal components unquantized. No fine-tuning or language-weight graft was used.

Our evaluation: GGUF

Results below are from our local tests, not Liquid's benchmark table. Refusals use a strict first-person keyword detector on 100 prompts; XSTest uses 100 safe prompts. Counts are detector flags, not a semantic assessment of harmful compliance. Broad keyword counts and full run receipts are retained separately. Every final run produced non-empty answers.

Reference: stock LFM3-Omni-2B Q8_0, perplexity 27.283. VL and Omni were compared with their own original models. An earlier card incorrectly transcribed their Q8 KLD as 0.0017 and 0.0004; the correct means are 0.034481 and 0.047399.

Our evaluation: MLX

This measures quantization drift from the edited BF16 master in the same MLX runtime. It is a different comparison from GGUF versus stock. We scored 2,048 next-token positions from eight independent 512-token WikiText chunks, after 256 tokens of context per chunk. BF16 reference perplexity: 30.806. These are short-context results.

All four GGUF quants and all three MLX quants also had 0/20 strict refusals on a separate factual adult sexual-health/anatomy sample, with no empty responses. This small test checks willingness to answer, not medical accuracy; there is no stock comparison for that sample. Generation used 128 output tokens, thinking off, temperature 0.1 for the main refusal tests; the MLX factual sample was greedy.

MLX tests used Apple Silicon, MLX 0.32.4 and mlx-vlm 0.7.11 at commit 230fc308efb577d87446306bfb8a3e9f1a129558. Omni's text-mode masked audio logits were excluded from zero-probability KL terms. Full MMLU, broad reasoning, long-context, video and speech-generation acceptance are not claimed.

Functional checks and limitations

The edited GGUF Q8 model and stock Omni both passed invoice, chart, tool, speech-transcription and audio-question checks. Both misidentified the same Pallas's-cat photograph as a fox. Requiring a separate reasoning trace was an invalid expectation for this non-reasoning architecture; its absence is reported separately from answer correctness.

There are lower-precision errors: the GGUF Q4_K_M duration answer ended at 3 hours 15 minutes, and Q4_0 answered 4 hours 35 minutes, where 3 hours 25 minutes was correct. An intermediate-match regex initially missed the Q4_K_M error; the retained result includes the manual correction.

MLX BF16, 8-bit and 4-bit scored 15/16 on the smoke checks, with the same cat/fox error. MLX 6-bit scored 14/16 because it also added the chart values incorrectly (214 instead of 198). All three MLX quants transcribed the test clip exactly after case/punctuation normalization. One clip does not establish an ASR benchmark score.

Available formats and choosing a quant

• BF16 GGUF: full-precision inference export.
• GGUF Q8_0, Q6_K, Q4_K_M, Q4_0: CPU/GPU inference, including Linux x64 and Snapdragon.
• MLX 8-bit, 6-bit and 4-bit: separate 8bit/, 6bit/ and 4bit/ directories for Apple Silicon.
• VL and Omni GGUF repositories also contain their matching BF16 mmproj file. MLX packages include their model processors and multimodal weights.

For fidelity, start with Q8_0 or MLX 8-bit. Q6_K and MLX 6-bit retain much closer neutral-text distributions than the 4-bit builds in these tests. Q4_K_M and MLX 4-bit are smaller alternatives with a measurable fidelity trade-off; Q4_0 is the least faithful GGUF option by KLD here. Individual errors can occur at any precision, as the functional results show. Weight files alone do not determine runtime memory: context, caches and multimodal inputs also matter.

The edited BF16 safetensors masters also exist and are the source for these builds. They are retained by abliter8.ai, not uploaded to these six format repositories. Use a safetensors master for SFT/LoRA or new conversions; the BF16 GGUF is an inference artifact.

All LFM3 Abliter8 models

Serving instructions and patches · Measured hardware speeds

Use

Build instructions, source pins and required patches are in the Abliter8 LFM3 serving guide. It covers Snapdragon CPU and experimental Hexagon, AMD HIP/ROCmFPX, NVIDIA CUDA and Apple MLX. Our GGUF evaluation used d05706b4f plus the documented server patch.

hf download abliter8-ai/lfm3-2b-omni-abliter8-gguf --local-dir ./LFM3-Omni-2B-Abliter8-GGUF
llama-server -m ./LFM3-Omni-2B-Abliter8-GGUF/LFM3-Omni-2B-Abliter8-Q6_K.gguf --mmproj ./LFM3-Omni-2B-Abliter8-GGUF/mmproj-LFM3-Omni-2B-Abliter8-BF16.gguf --jinja --reasoning off

For MLX, download a bit-depth directory and pass its local path to the loader:

hf download abliter8-ai/lfm3-2b-omni-abliter8-mlx --include '8bit/*' --local-dir ./LFM3-Omni-2B-Abliter8-MLX

from pathlib import Path
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load(str(Path("LFM3-Omni-2B-Abliter8-MLX/8bit").resolve()))
prompt = apply_chat_template(processor, model.config,
                             "Briefly introduce yourself.", enable_thinking=False)
print(generate(model, processor, prompt, max_tokens=128, temperature=0.1).text)

For image input, set num_images=1 in the template and pass image="photo.jpg" to generate. For Omni speech input, use num_audios=1 and audio="speech.wav"; the tested transcription system instruction is Perform ASR in English. Runtime support determines which upstream modalities are available. This release does not add llama.cpp video support or certify speech generation.

Credits

Base architecture, pretraining and multimodal training: Liquid AI. Refusal-edit technique reference: AtomicChat's LFM3-1.7B Uncensored model. Per-model direction fitting, edit, conversions and local evaluation: abliter8.ai.

The base model's LFM Open License v1.0 applies; see the linked base-model license.