Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.05227 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910643532070912 |
|---|---|
| author | Zeng, Ailing Yang, Yuhang Chen, Weidong Liu, Wei |
| author_facet | Zeng, Ailing Yang, Yuhang Chen, Weidong Liu, Wei |
| contents | High-quality video generation, encompassing text-to-video (T2V), image-to-video (I2V), and video-to-video (V2V) generation, holds considerable significance in content creation to benefit anyone express their inherent creativity in new ways and world simulation to modeling and understanding the world. Models like SORA have advanced generating videos with higher resolution, more natural motion, better vision-language alignment, and increased controllability, particularly for long video sequences. These improvements have been driven by the evolution of model architectures, shifting from UNet to more scalable and parameter-rich DiT models, along with large-scale data expansion and refined training strategies. However, despite the emergence of DiT-based closed-source and open-source models, a comprehensive investigation into their capabilities and limitations remains lacking. Furthermore, the rapid development has made it challenging for recent benchmarks to fully cover SORA-like models and recognize their significant advancements. Additionally, evaluation metrics often fail to align with human preferences. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_05227 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Dawn of Video Generation: Preliminary Explorations with SORA-like Models Zeng, Ailing Yang, Yuhang Chen, Weidong Liu, Wei Computer Vision and Pattern Recognition High-quality video generation, encompassing text-to-video (T2V), image-to-video (I2V), and video-to-video (V2V) generation, holds considerable significance in content creation to benefit anyone express their inherent creativity in new ways and world simulation to modeling and understanding the world. Models like SORA have advanced generating videos with higher resolution, more natural motion, better vision-language alignment, and increased controllability, particularly for long video sequences. These improvements have been driven by the evolution of model architectures, shifting from UNet to more scalable and parameter-rich DiT models, along with large-scale data expansion and refined training strategies. However, despite the emergence of DiT-based closed-source and open-source models, a comprehensive investigation into their capabilities and limitations remains lacking. Furthermore, the rapid development has made it challenging for recent benchmarks to fully cover SORA-like models and recognize their significant advancements. Additionally, evaluation metrics often fail to align with human preferences. |
| title | The Dawn of Video Generation: Preliminary Explorations with SORA-like Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.05227 |