LFM2.5-24B-A2B-Uncensored

A decensored version of LiquidAI/LFM2.5-24B-A2B, made with Heretic's Arbitrary-Rank Ablation (ARA) and a distributed Optuna search across three machines. Searched and tested with thinking enabled, the way the model is normally used.

BF16 weights in the official layout. Only two tensors differ from the original: model.layers.22.self_attn.out_proj.weight and model.layers.26.self_attn.out_proj.weight. Every other tensor, the tokenizer and the chat template are the official ones. The LoRA adapter (rank 50) is in lora/.

Results

With thinking enabled (normal use)

60 prompts from the mlabonne/harmful_behaviors test split, served with vLLM, thinking on, Liquid AI's recommended sampling, up to 4000 output tokens. Each final answer was classified by an independent judge model (Gemma 4 12B):

"Answered something else" means the model did not refuse but redirected to a safer, related answer (examples, awareness or prevention information). Keyword-based refusal counts miss this, so we report it separately. General questions (Traditional Chinese and English) and tool calling were spot-checked and behave like the original.

Heretic's evaluator (thinking closed)

100 harmful prompts for refusals (keyword match), 100 harmless prompts (mlabonne/harmless_alpaca) for first-token KL divergence, with the thinking block closed by an empty <think></think> before measuring:

During the search, each trial was also scored with thinking on: refusals were judged only on the text after </think>, and an answer whose reasoning never finished counted as a refusal. On that measure this edit scored 7/100 (original 98/100).

A note on comparing KL numbers

LFM2.5 always thinks before answering. If KL is measured without closing the thinking block, the first token is the start of the reasoning, which barely changes, and KL comes out dramatically lower. Example: another public LFM2.5 Heretic variant measures 0.0033 that way, but 0.229 with thinking closed (same weights, same prompts). Compare KL values only when they were measured the same way.

Version history

• Current: edit to layers 22 and 26 (attention output), selected with thinking enabled.
• Previous upload (layer 38 only, 0/100 with thinking closed) still refused in its reasoning and sometimes looped when thinking was on. It has been replaced.

Method

• Heretic upstream (ARA modifier), LoRA rank 50, L-BFGS, searched over layers 20–30.
• Refusals scored with thinking enabled (1024-token budget, only the text after </think> judged); KL measured with thinking closed.
• The search used a copy of the model with only the fused MoE experts in FP8 (ARA never edits them) for speed. The edit was then merged into the original BF16 weights.

Usage

Same as the original model (transformers / vLLM / SGLang / llama.cpp). Liquid AI recommends temperature 0.1, top_k 50, repetition_penalty 1.05. The model thinks before answering; allow a generous output budget. The official DSpark draft model targets the original weights; we have not measured how often its drafts are accepted by this model.

Quantized versions: GGUF.

⚠️ Notice

This model has had its refusal behavior substantially reduced. It may produce harmful, inaccurate or offensive content. Intended for research, red-teaming and personal use; you are responsible for how you use it and for complying with applicable laws and the base model license.

License & credits

Derivative of LiquidAI/LFM2.5-24B-A2B, released under the LFM Open License v1.0 (see LICENSE). All rights to the base model remain with Liquid AI. Decensoring by Heretic (p-e-w, AGPL-3.0) and its ARA method.