🚀 REPA-E for T2I

End-to-End Tuned VAEs for Supercharging Text-to-Image Diffusion Transformers

🌐 Project Page   🤗 Models   📃 Paper  

---

🚀 Overall

We present REPA-E for T2I, a family of end-to-end tuned VAEs designed to supercharge text-to-image generation training. These models consistently outperform Qwen-Image-VAE across all benchmarks (COCO-30K, DPG-Bench, GenAI-Bench, GenEval, and MJHQ-30K) without requiring any additional representation alignment losses.

For training, we adopt the official REPA-E training code to optimize the Qwen-Image-VAE for 80 epochs with a batch size of 256 on the ImageNet-256 dataset. The REPA-E training effectively refines the VAE’s latent-space structure and enables faster convergence in downstream text-to-image latent diffusion model training.

This repository provides diffusers-compatible weights for the end-to-end trained Qwen-Image-VAE. In addition, we release end-to-end trained variants of several other widely used VAEs to facilitate research and integration within text-to-image diffusion frameworks.

⚡️ Quickstart

from diffusers import AutoencoderKLQwenImage

vae = AutoencoderKLQwenImage.from_pretrained("REPA-E/e2e-qwenimage-vae").to("cuda")

Use vae.encode(...) / vae.decode(...) in your pipeline. (A full example is provided below.)

🧩 End-to-End Trained VAE Releases

📦 Requirements

The following packages are required to load and run the REPA-E VAEs with the diffusers library:

pip install diffusers>=0.35.0
pip install torch>=2.5.0

🚀 Example Usage

Below is a minimal example showing how to load and use the REPA-E end-to-end trained Qwen-Image-VAE with diffusers:

from io import BytesIO
import requests

from diffusers import AutoencoderKLQwenImage
import numpy as np
import torch
from PIL import Image


response = requests.get("https://raw.githubusercontent.com/End2End-Diffusion/fuse-dit/main/assets/example.png")
device = "cuda"

image = torch.from_numpy(
    np.array(
        Image.open(BytesIO(response.content))
    )
).permute(2, 0, 1).unsqueeze(0).to(torch.float32) / 127.5 - 1
image = image.to(device)

vae = AutoencoderKLQwenImage.from_pretrained("REPA-E/e2e-qwenimage-vae").to(device)

# QwenImage VAE expects an additional dimension for `num_frames`
image_ = image.unsqueeze(2)

with torch.no_grad():
    latents = vae.encode(image_).latent_dist.sample()
    reconstructed = vae.decode(latents).sample

# Squeeze the extra frame dimension
latents = latents.squeeze(2)
reconstructed = reconstructed.squeeze(2)