Wanxiao 2.1-1.3B-LoRA-Aesthetic-Distillation-v1

Model Introduction

This LoRA model is trained based on the Wanxiao 2.1-1.3B model and the DiffSynth-Studio framework. It has been fine-tuned on an aesthetically curated dataset to enhance the visual appeal of generated videos, and supports disabling classifier-free guidance for faster inference.

Recommended parameter settings:

1. cfg_scale=1
2. sigma_shift=10

Note: Using this model may reduce the diversity of generated videos. Adjust the lora_alpha value to control the influence of the LoRA on generation results.

Model Performance

Below are comparisons of model outputs under the settings cfg_scale=1 and sigma_shift=10.

Prompt: A serene tropical beach at sunset, with palm trees, flower-covered rocky cliffs, and gentle waves breaking

Prompt: A robot in a desert world, covered in rust, with dappled sunlight

Usage Instructions

This model is built on the DiffSynth-Studio framework. Please install it first:

pip install diffsynth

import torch
from diffsynth import ModelManager, WanVideoPipeline, save_video
from modelscope import snapshot_download

snapshot_download(
    model_id="DiffSynth-Studio/Wan2.1-1.3b-lora-aesthetics-v1",
    local_dir="models/DiffSynth-Studio/Wan2.1-1.3b-lora-aesthetics-v1",
    allow_file_pattern="*.safetensors"
)
model_manager = ModelManager(device="cpu")
model_manager.load_models(
    [
        "models/Wan-AI/Wan2.1-T2V-1.3B/diffusion_pytorch_model.safetensors",
        "models/Wan-AI/Wan2.1-T2V-1.3B/models_t5_umt5-xxl-enc-bf16.pth",
        "models/Wan-AI/Wan2.1-T2V-1.3B/Wan2.1_VAE.pth",
    ],
    torch_dtype=torch.bfloat16,
)
model_manager.load_lora("models/DiffSynth-Studio/Wan2.1-1.3b-lora-aesthetics-v1/model.safetensors", lora_alpha=1)
pipe = WanVideoPipeline.from_model_manager(model_manager, torch_dtype=torch.bfloat16, device="cuda")
pipe.enable_vram_management(num_persistent_param_in_dit=None)

video = pipe(
    prompt="A tranquil tropical beach scene at sunset, with palm trees, flower-covered rocky cliffs, and gentle waves",
    num_inference_steps=50,
    seed=1,
    tiled=True,
    cfg_scale=1,
    sigma_shift=10,
)
save_video(video, "video.mp4", fps=15, quality=5)