🤗 KS-Gen Model Checkpoints

📋 Overview

The KS-Gen framework decomposes complex human skills into key steps and generates video clips for each step, rather than creating full-length videos. This approach enables more precise control over the generated content and better captures the dynamics of complex human behavior.

Our model checkpoints include:

Video Generation Models: These models take key-step descriptions and generate corresponding video clips. They include state-of-the-art architectures like DynamiCrafter, AnimateAnything, and Stable Video Diffusion, specifically fine-tuned for key-step generation tasks.
Image Generation Models: These models generate reference images for each key step, which serve as initial frames for the video generation process. They include Stable Diffusion XL with specialized components like our Skill Transformer that conditions the generation on key-step descriptions.

📦 Checkpoint Files

Video Generation Models

Model	File
DynamiCrafter	`dynamicrafter_1024.ckpt`
AnimateAnything	`animateanything.ckpt`
Stable Video Diffusion	`svd.ckpt`

Image Generation Models

Model	File
Stable Diffusion XL (Projection)	`sdxl_proj.ckpt`
Stable Diffusion XL (Skill Transformer)	`sdxl_skill_transformer.ckpt`

🚀 Usage

For detailed usage instructions, please refer to the official repository: https://github.com/MCG-NJU/KS-Gen

📝 Citation

If you use these models in your research, please cite our paper:

@article{wu2025learning,
  title={Learning Human Skill Generators at Key-Step Levels},
  author={Wu, Yilu and Zhu, Chenhui and Wang, Shuai and Wang, Hanlin and Wang, Jing and Zhang, Zhaoxiang and Wang, Limin},
  journal={arXiv preprint arXiv:2502.08234},
  year={2025},
  url={https://arxiv.org/abs/2502.08234}, 
}

⚖️ License

MIT License - see the LICENSE file for details.

📬 Contact

For questions or issues related to the models, please open an issue on the repository.