Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909192905818112 |
|---|---|
| author | Bao, Fan Xiang, Chendong Yue, Gang He, Guande Zhu, Hongzhou Zheng, Kaiwen Zhao, Min Liu, Shilong Wang, Yaole Zhu, Jun |
| author_facet | Bao, Fan Xiang, Chendong Yue, Gang He, Guande Zhu, Hongzhou Zheng, Kaiwen Zhao, Min Liu, Shilong Wang, Yaole Zhu, Jun |
| contents | We introduce Vidu, a high-performance text-to-video generator that is capable of producing 1080p videos up to 16 seconds in a single generation. Vidu is a diffusion model with U-ViT as its backbone, which unlocks the scalability and the capability for handling long videos. Vidu exhibits strong coherence and dynamism, and is capable of generating both realistic and imaginative videos, as well as understanding some professional photography techniques, on par with Sora -- the most powerful reported text-to-video generator. Finally, we perform initial experiments on other controllable video generation, including canny-to-video generation, video prediction and subject-driven generation, which demonstrate promising results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_04233 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models Bao, Fan Xiang, Chendong Yue, Gang He, Guande Zhu, Hongzhou Zheng, Kaiwen Zhao, Min Liu, Shilong Wang, Yaole Zhu, Jun Computer Vision and Pattern Recognition Machine Learning We introduce Vidu, a high-performance text-to-video generator that is capable of producing 1080p videos up to 16 seconds in a single generation. Vidu is a diffusion model with U-ViT as its backbone, which unlocks the scalability and the capability for handling long videos. Vidu exhibits strong coherence and dynamism, and is capable of generating both realistic and imaginative videos, as well as understanding some professional photography techniques, on par with Sora -- the most powerful reported text-to-video generator. Finally, we perform initial experiments on other controllable video generation, including canny-to-video generation, video prediction and subject-driven generation, which demonstrate promising results. |
| title | Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2405.04233 |