CascadeV: An Implementation of Wurstchen Architecture for Video Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912206936866816 |
|---|---|
| author | Lin, Wenfeng Wei, Jiangchuan Liu, Boyuan Zhang, Yichen Yan, Shiyue Guo, Mingyu |
| author_facet | Lin, Wenfeng Wei, Jiangchuan Liu, Boyuan Zhang, Yichen Yan, Shiyue Guo, Mingyu |
| contents | Recently, with the tremendous success of diffusion models in the field of text-to-image (T2I) generation, increasing attention has been directed toward their potential in text-to-video (T2V) applications. However, the computational demands of diffusion models pose significant challenges, particularly in generating high-resolution videos with high frame rates. In this paper, we propose CascadeV, a cascaded latent diffusion model (LDM), that is capable of producing state-of-the-art 2K resolution videos. Experiments demonstrate that our cascaded model achieves a higher compression ratio, substantially reducing the computational challenges associated with high-quality video generation. We also implement a spatiotemporal alternating grid 3D attention mechanism, which effectively integrates spatial and temporal information, ensuring superior consistency across the generated video frames. Furthermore, our model can be cascaded with existing T2V models, theoretically enabling a 4$\times$ increase in resolution or frames per second without any fine-tuning. Our code is available at https://github.com/bytedance/CascadeV. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_16612 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CascadeV: An Implementation of Wurstchen Architecture for Video Generation Lin, Wenfeng Wei, Jiangchuan Liu, Boyuan Zhang, Yichen Yan, Shiyue Guo, Mingyu Computer Vision and Pattern Recognition Recently, with the tremendous success of diffusion models in the field of text-to-image (T2I) generation, increasing attention has been directed toward their potential in text-to-video (T2V) applications. However, the computational demands of diffusion models pose significant challenges, particularly in generating high-resolution videos with high frame rates. In this paper, we propose CascadeV, a cascaded latent diffusion model (LDM), that is capable of producing state-of-the-art 2K resolution videos. Experiments demonstrate that our cascaded model achieves a higher compression ratio, substantially reducing the computational challenges associated with high-quality video generation. We also implement a spatiotemporal alternating grid 3D attention mechanism, which effectively integrates spatial and temporal information, ensuring superior consistency across the generated video frames. Furthermore, our model can be cascaded with existing T2V models, theoretically enabling a 4$\times$ increase in resolution or frames per second without any fine-tuning. Our code is available at https://github.com/bytedance/CascadeV. |
| title | CascadeV: An Implementation of Wurstchen Architecture for Video Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.16612 |