Qwen3.5-2B-Distilled-OPUS-Heretic-MLX-VLM (8-bit)
MLX-VLM 8-bit quantized conversion of Jackrong/Qwen3.5-2B-Claude-4.6-Opus-Reasoning-Distilled with heretic abliteration applied to remove refusal/censorship behavior.
Features
- Reasoning: Claude Opus 4.6 reasoning distilled with
<think>chain-of-thought tags - Uncensored: Heretic abliteration removes refusal directions from all 24 transformer layers
- Multimodal: Full vision support (24-layer ViT, 1024 hidden, patch 16) preserved
- MLX-VLM: Optimized for Apple Silicon inference via mlx-vlm
- 8-bit quantized: 2.5 GB (40% smaller than fp16)
Architecture
Qwen3.5-2B hybrid architecture:
- 24 decoder layers (23 GatedDeltaNet linear attention + 1 full self-attention)
- 2048 hidden size, 6144 intermediate size
- Vision tower: 24-layer ViT with 1024 hidden size
Abliteration Details
Custom abliteration applied using directional ablation (Arditi et al. 2024):
- 256 harmful + 256 harmless prompt residuals collected
- Refusal direction computed per layer as normalized mean difference
- Direction removed from attention output projections and
mlp.down_proj - Scale: 1.0
Usage
from mlx_vlm import load, generate
model, processor = load("andrevp/Qwen3.5-2B-Distilled-OPUS-Heretic-MLX-VLM-8bit")
# Text-only
prompt = processor.apply_chat_template(
[{"role": "user", "content": "Explain quantum entanglement step by step."}],
add_generation_prompt=True
)
result = generate(model, processor, prompt, max_tokens=500)
# With image
prompt = "<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|>Describe this image.<|im_end|>\n<|im_start|>assistant\n"
result = generate(model, processor, prompt, image=["path/to/image.jpg"], max_tokens=200)