Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.00131 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915041973895168 |
|---|---|
| author | Lin, Bin Ge, Yunyang Cheng, Xinhua Li, Zongjian Zhu, Bin Wang, Shaodong He, Xianyi Ye, Yang Yuan, Shenghai Chen, Liuhan Jia, Tanghui Zhang, Junwu Tang, Zhenyu Pang, Yatian She, Bin Yan, Cen Hu, Zhiheng Dong, Xiaoyi Chen, Lin Pan, Zhang Zhou, Xing Dong, Shaoling Tian, Yonghong Yuan, Li |
| author_facet | Lin, Bin Ge, Yunyang Cheng, Xinhua Li, Zongjian Zhu, Bin Wang, Shaodong He, Xianyi Ye, Yang Yuan, Shenghai Chen, Liuhan Jia, Tanghui Zhang, Junwu Tang, Zhenyu Pang, Yatian She, Bin Yan, Cen Hu, Zhiheng Dong, Xiaoyi Chen, Lin Pan, Zhang Zhou, Xing Dong, Shaoling Tian, Yonghong Yuan, Li |
| contents | We introduce Open-Sora Plan, an open-source project that aims to contribute a large generation model for generating desired high-resolution videos with long durations based on various user inputs. Our project comprises multiple components for the entire video generation process, including a Wavelet-Flow Variational Autoencoder, a Joint Image-Video Skiparse Denoiser, and various condition controllers. Moreover, many assistant strategies for efficient training and inference are designed, and a multi-dimensional data curation pipeline is proposed for obtaining desired high-quality data. Benefiting from efficient thoughts, our Open-Sora Plan achieves impressive video generation results in both qualitative and quantitative evaluations. We hope our careful design and practical experience can inspire the video generation research community. All our codes and model weights are publicly available at \url{https://github.com/PKU-YuanGroup/Open-Sora-Plan}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_00131 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Open-Sora Plan: Open-Source Large Video Generation Model Lin, Bin Ge, Yunyang Cheng, Xinhua Li, Zongjian Zhu, Bin Wang, Shaodong He, Xianyi Ye, Yang Yuan, Shenghai Chen, Liuhan Jia, Tanghui Zhang, Junwu Tang, Zhenyu Pang, Yatian She, Bin Yan, Cen Hu, Zhiheng Dong, Xiaoyi Chen, Lin Pan, Zhang Zhou, Xing Dong, Shaoling Tian, Yonghong Yuan, Li Computer Vision and Pattern Recognition Artificial Intelligence We introduce Open-Sora Plan, an open-source project that aims to contribute a large generation model for generating desired high-resolution videos with long durations based on various user inputs. Our project comprises multiple components for the entire video generation process, including a Wavelet-Flow Variational Autoencoder, a Joint Image-Video Skiparse Denoiser, and various condition controllers. Moreover, many assistant strategies for efficient training and inference are designed, and a multi-dimensional data curation pipeline is proposed for obtaining desired high-quality data. Benefiting from efficient thoughts, our Open-Sora Plan achieves impressive video generation results in both qualitative and quantitative evaluations. We hope our careful design and practical experience can inspire the video generation research community. All our codes and model weights are publicly available at \url{https://github.com/PKU-YuanGroup/Open-Sora-Plan}. |
| title | Open-Sora Plan: Open-Source Large Video Generation Model |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2412.00131 |