VideoAuteur: Towards Long Narrative Video Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916783317843968 |
|---|---|
| author | Xiao, Junfei Cheng, Feng Qi, Lu Gui, Liangke Cen, Jiepeng Ma, Zhibei Yuille, Alan Jiang, Lu |
| author_facet | Xiao, Junfei Cheng, Feng Qi, Lu Gui, Liangke Cen, Jiepeng Ma, Zhibei Yuille, Alan Jiang, Lu |
| contents | Recent video generation models have shown promising results in producing high-quality video clips lasting several seconds. However, these models face challenges in generating long sequences that convey clear and informative events, limiting their ability to support coherent narrations. In this paper, we present a large-scale cooking video dataset designed to advance long-form narrative generation in the cooking domain. We validate the quality of our proposed dataset in terms of visual fidelity and textual caption accuracy using state-of-the-art Vision-Language Models (VLMs) and video generation models, respectively. We further introduce a Long Narrative Video Director to enhance both visual and semantic coherence in generated videos and emphasize the role of aligning visual embeddings to achieve improved overall video quality. Our method demonstrates substantial improvements in generating visually detailed and semantically aligned keyframes, supported by finetuning techniques that integrate text and image embeddings within the video generation process. Project page: https://videoauteur.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_06173 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VideoAuteur: Towards Long Narrative Video Generation Xiao, Junfei Cheng, Feng Qi, Lu Gui, Liangke Cen, Jiepeng Ma, Zhibei Yuille, Alan Jiang, Lu Computer Vision and Pattern Recognition Recent video generation models have shown promising results in producing high-quality video clips lasting several seconds. However, these models face challenges in generating long sequences that convey clear and informative events, limiting their ability to support coherent narrations. In this paper, we present a large-scale cooking video dataset designed to advance long-form narrative generation in the cooking domain. We validate the quality of our proposed dataset in terms of visual fidelity and textual caption accuracy using state-of-the-art Vision-Language Models (VLMs) and video generation models, respectively. We further introduce a Long Narrative Video Director to enhance both visual and semantic coherence in generated videos and emphasize the role of aligning visual embeddings to achieve improved overall video quality. Our method demonstrates substantial improvements in generating visually detailed and semantically aligned keyframes, supported by finetuning techniques that integrate text and image embeddings within the video generation process. Project page: https://videoauteur.github.io/ |
| title | VideoAuteur: Towards Long Narrative Video Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.06173 |