In-Context Editing LoRA (Qwen-Image-Edit-2511)

Model Introduction

This is an interesting model that enables Qwen-Image-Edit-2511 with In-Context Editing capabilities. You can input three images to the model: Image 1, Image 2, and Image 3, and the model will automatically apply the transformation from Image 1 to Image 2 onto Image 3.

For more details about training strategies and implementation, feel free to check our technical blog.

Demo Results

Image sources:

• Input Image 1: https://modelscope.cn/aigc/imageGeneration?tab=advanced&imageId=18968195
• Input Image 2: Generated via single-image editing using the editing model
• Input Image 3: https://modelscope.cn/aigc/imageGeneration?tab=advanced&imageId=18723032

Prompt: Edit image 3 based on the transformation from image 1 to image 2.

Negative prompt: yellowish tint, AI-like appearance, unrealistic, ugly, oily skin, abnormal limbs, disproportionate limbs

• Example 1: Expression reference

• Example 2: Style transfer

• Example 3: Adding entities

• Example 4: Local editing

Inference Code

Install DiffSynth-Studio:

git clone https://github.com/modelscope/DiffSynth-Studio.git  
cd DiffSynth-Studio
pip install -e .

Inference code:

from diffsynth.pipelines.qwen_image import QwenImagePipeline, ModelConfig
from modelscope import snapshot_download
from PIL import Image
import torch

Load models

pipe = QwenImagePipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(model_id="Qwen/Qwen-Image-Edit-2511", origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="text_encoder/model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="vae/diffusion_pytorch_model.safetensors"),
    ],
    processor_config=ModelConfig(model_id="Qwen/Qwen-Image-Edit", origin_file_pattern="processor/"),
)
lora = ModelConfig(
    model_id="DiffSynth-Studio/Qwen-Image-Edit-2511-ICEdit-LoRA",
    origin_file_pattern="model.safetensors"
)
pipe.load_lora(pipe.dit, lora)

Load images

snapshot_download(
    "DiffSynth-Studio/Qwen-Image-Edit-2511-ICEdit-LoRA",
    local_dir="./data",
    allow_file_pattern="assets/*"
)
edit_image = [
    Image.open("data/assets/image1_original.png"),
    Image.open("data/assets/image1_edit_1.png"),
    Image.open("data/assets/image2_original.png")
]
prompt = "Edit image 3 based on the transformation from image 1 to image 2."
negative_prompt = "yellowish, AI-looking, unrealistic, ugly, oily skin, abnormal limbs, disproportionate limbs"

Generate

image_4 = pipe(
    prompt=prompt, negative_prompt=negative_prompt,
    edit_image=edit_image,
    seed=1,
    num_inference_steps=50,
    height=1280,
    width=720,
    zero_cond_t=True,
)
image_4.save("image.png")