SparkWan2.1-T2V-14B-720P-0.97Sparsity

This repository contains the released SparkDiffusion checkpoint for Wan2.1 T2V 14B at 720P. It supports attention sparsity from 90% to 97% (topk=0.1 to topk=0.03).

SparkDiffusion uses RoLA for learned sparse attention and CrossDistill for few-step diffusion distillation, reducing video generation latency while retaining the original Wan text-to-video task and model architecture.

Checkpoint

• SparkWan2.1-T2V-14B-720P-0.97Sparsity.pth
• Base model: Wan-AI/Wan2.1-T2V-14B
• Task: text-to-video
• Resolution: 720P
• Inference steps: 4
• Supported attention sparsity: 90%–97%

Project

• Homepage: https://sparkdiffusion.github.io/
• GitHub: https://github.com/AlibabaResearch/SparkDiffusion

Citation

@misc{liu2026sparkdiffusionmitigatinghighsparsitytrap,
  title={SparkDiffusion: Mitigating the High-Sparsity Trap --- A Unified Framework for up to $265\times$ Single-GPU Acceleration of Visual Generation},
  author={Yuxi Liu and Haoyu Li and Zekun Zhang and Tengxu Sun and Yixiang Cai and Jiayong Li and Yifei Xia and Tianle Liu and Baole Ai and Ang Wang and Jiamang Wang and Lin Qu and Kai Zhang and Kun Yuan and Bin Cui},
  year={2026},
  eprint={2609.23153},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2609.23153},
}

@misc{zhang2026rolarotarypositionedlowranklinear,
  title={RoLA: Rotary-Positioned Low-Rank Linear Attention for Efficient Diffusion Transformers},
  author={Zekun Zhang and Yixiang Cai and Yuxi Liu and Tengxu Sun and Tianle Liu and Zhoutong Wu and Haoyu Li and Baole Ai and Ang Wang and Jiamang Wang and Lin Qu and Kun Yuan},
  year={2026},
  eprint={2609.06712},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2609.06712},
}

@misc{liu2026crossdistillbalancingqualitydiversity,
  title={CrossDistill: Balancing Quality and Diversity via Trajectory-Level Hybrid Few-Step Distillation},
  author={Yuxi Liu and Haoyu Li and Yixiang Cai and Tengxu Sun and Zekun Zhang and Baole Ai and Ang Wang and Jiamang Wang and Lin Qu and Kun Yuan and Kai Zhang},
  year={2026},
  eprint={2609.14725},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2609.14725},
}

@misc{liu2026ropeslr3dropedrivensparselowrank,
  title={RoPeSLR: 3D RoPE-driven Sparse-LowRank Attention for Efficient Diffusion Transformers},
  author={Yuxi Liu and Zekun Zhang and Yixiang Cai and Renjia Deng and Yutong He and Kun Yuan},
  year={2026},
  eprint={2605.20659},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2605.20659},
}