Semantic VAE
Purpose: Extremely fast convergence on illustraton data for a downstream DiT model
Trained from DINOv2 in 12 hours as follows:
Frozen DINOv2 -> LayerNorm -> 32 channels
Frozen DINOv2 with patch embed unfrozen -> LayerNorm -> 32 channels
64 channel bottleneck with Sigreg loss (1e-3 weight) from LeJEPA
VA-VAE style convolution decoder
Loss is all layers of DINOv2 and DINOv3 equally weighted MSE on input vs reconstruction.
There are no pixel losses/LPIPS/GAN used, this is DINO space loss only.