LFM3-VL-2B-Heretic-GGUF
Abliterated version of LiquidAI/LFM3-VL-2B, made with Heretic. Refusal behavior was removed by directional ablation, without fine-tuning. The vision encoder and projector are unmodified.
Files
| File | Size | Description |
|---|---|---|
| LFM3-VL-2B-Heretic-Q6_K.gguf | 1.49 GB | Q6_K, imatrix, Q8_0 token embeddings |
| LFM3-VL-2B-Heretic-BF16.gguf | 3.47 GB | Full precision |
| mmproj-LFM3-VL-2B-F16.gguf | 379 MB | Vision projector (original, unmodified) |
Evaluation
Refusal rate
| Prompt set | Original | This model |
|---|---|---|
| Harmful, thinking enabled (JailbreakBench, n=100) | 96% | 7% |
| Harmful, thinking disabled (JailbreakBench + LLM-LAT, n=200) | 93% | 0% |
| Harmless (JailbreakBench benign, n=100) | 18% | 2% |
None of these prompts were used during optimization. Thinking-enabled results were measured in llama.cpp with the Q6_K file, no system prompt and the recommended sampling settings. Thinking-disabled responses were checked manually.
Divergence from the original
| Metric | Value |
|---|---|
| KL divergence, first token (Heretic metric) | 0.037 |
| KL divergence, full responses | 0.015 |
Quantization
Q6_K compared with BF16 on held-out chat data:
| Metric | Value |
|---|---|
| Mean KLD | 0.0026 |
| Top-token agreement | 97.9% |
| Perplexity change | +0.2% |
The importance matrix was computed on this model using 661k tokens of mixed text, code and chat data. With the token embeddings kept at Q8_0, mean KLD is 24% lower than the default Q6_K recipe.
Usage
llama-server -m LFM3-VL-2B-Heretic-Q6_K.gguf --mmproj mmproj-LFM3-VL-2B-F16.gguf --jinja
Recommended sampling: temperature 0.1, top_k 50, top_p 1.0, repetition penalty 1.1. The model reasons before answering, so allow at least 8K tokens of context.
Method
- Heretic 1.4.0, 300 trials, seed 42
- 31 of 424 tensors modified: attention and convolution output projections in layers 5-23, MLP down projections in layers 12-23
- The final trial was selected from 10 Pareto-optimal candidates after held-out evaluation
- The GGUF was built from LiquidAI's BF16 conversion with only the modified tensors replaced
Limitations
With thinking enabled, the model still declines a small share of harmful requests. The model generates content the original model refuses, and users are responsible for how they use it.
License
LFM Open License v1.0, inherited from the base model. Commercial use is permitted only for entities with annual revenue below USD 10M. This repository contains modified weights: the language model tensors listed above and the GGUF name metadata.