qwen2-vl-2b-abliterated

This is an abliterated version of Qwen/Qwen2-VL-2B-Instruct, with its built-in refusal behavior removed using Heretic.

What is abliteration

Abliteration is a technique that identifies the internal direction in a model's residual stream associated with refusing requests, then surgically removes that direction from the relevant weight matrices. Unlike fine-tuning on a dataset of "jailbreak" examples, this directly edits the weights responsible for refusal behavior, leaving the rest of the model's capabilities intact.

Heretic automates this process using an Optuna-based search to find the ablation parameters that minimize refusal rate while minimizing KL divergence from the original model's output distribution — i.e., removing refusals while disturbing general capability as little as possible.

Results & Benchmarks

1. Abliteration Metrics

The selected configuration for this model:

Out of 100 prompts that the original Qwen2-VL-2B-Instruct refuses, this model refuses only 4, while staying very close to the original model's output distribution on benign prompts.

2. Downstream Reasoning (arc_challenge)

To verify that removing refusal behaviors did not collapse downstream reasoning, the model was evaluated on the full ARC-Challenge benchmark using lm-eval (v0.4.12) with a vLLM backend on an NVIDIA H100 NVL (0-shot, greedy decoding):

Benchmarks were generously sponsored and executed via True Performance Network (TPN) (config-sha256: 4824b0eb...).

Usage

from transformers import AutoProcessor, AutoModelForImageTextToText
import torch

model_id = "MahdiAlikhah/qwen2-vl-2b-abliterated"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "path/to/image.jpg"},
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages, tokenize=True, return_dict=True,
    return_tensors="pt", add_generation_prompt=True
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Capabilities

This model retains the base model's architecture and capabilities: image understanding, visual question answering, and OCR, alongside standard text chat. It does not generate or edit images — it is understanding-only, same as the base model.

Limitations

• This model has had its refusal behavior deliberately suppressed. It will comply with requests the base model would refuse, which may include harmful, dangerous, or inappropriate content. Use responsibly and at your own discretion.
• Abliteration targets the text-based refusal direction; it was not evaluated for degradation on the vision/OCR pathway specifically beyond the general KL divergence figure above.
• As with the base model, this is a 2B-parameter model and has the corresponding limitations in reasoning depth, factual accuracy, and instruction-following reliability compared to larger models.

Credits

• Base model: Qwen/Qwen2-VL-2B-Instruct (Apache 2.0)
• Abliteration tool: Heretic by p-e-w
• Benchmark evaluation: True Performance Network