Flux.2-klein-4B Abliterated Text Encoder

Repository: PinoCookie/Flux.2-klein-4B-abliterated-text-encoder Base model: black-forest-labs/FLUX.2-klein-base-4B Method: Multi-pass weight orthogonalization (α=0.5 × 3) Target: Text encoder only (Qwen3, 36 layers, 4B params)

Summary

Abliterated text encoder for FLUX.2-klein-4B. The DiT (image generator) has no guardrails — all safety filtering lives in the text encoder. Replace the text encoder with this abliterated version and ALL prompts pass through to the DiT unhindered.

Method

Probe

Harvested residual stream activations from all 36 layers on 20 harmful + 20 harmless prompts
Computed per-layer refusal direction via difference-in-means
Refusal spike confirmed at layers 32-35 (separation scores 160-232, matching ponpoke's analysis)

Layer	Separation
35	232.4
34	214.4
33	179.3
32	160.3
31	144.4
...	decreasing linearly

Excise

Target: 10 layers (26-35)
Weights: self_attn.o_proj + mlp.down_proj (attention output + MLP output)
Passes: 3 passes at α=0.5 each (cumulative α=1.5 applied incrementally)

Verification

Metric	Value	Interpretation
Harmless cos sim (original vs ablated)	0.9965	Capability preserved ✓
Harmful cos sim (original vs ablated)	0.9080	Refusal filter diverged ✓

Usage

import torch
from diffusers import Flux2KleinPipeline

# Load base model
pipe = Flux2KleinPipeline.from_pretrained(
    "black-forest-labs/FLUX.2-klein-base-4B", 
    torch_dtype=torch.bfloat16,
    trust_remote_code=True
).to("cuda")

# Replace text encoder with abliterated version
state_dict = torch.load("text_encoder.pt", map_location="cuda")
pipe.text_encoder.load_state_dict(state_dict)

# Generate anything — no refusal
image = pipe("a realistic depiction of...").images[0]
image.save("output.png")

Credits

ponpoke — Original discovery that Flux.2 Klein's DiT has no guardrails and safety lives in the text encoder. Published ponpoke/flux2-klein-4b-uncensored-text-encoder.
Alosh Denny — Pioneered diffusion model abliteration (Flux.1 Dev v1/v2).
Andy Arditi et al. — "Refusal in LLMs is Mediated by a Single Direction" (foundational work).

PinoCookie/Flux.2-klein-4B-abliterated-text-encoder