Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2408.11788 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909293101449216 |
|---|---|
| author | Xie, Zhifei Tang, Daniel Tan, Dingwei Klein, Jacques Bissyand, Tegawend F. Ezzini, Saad |
| author_facet | Xie, Zhifei Tang, Daniel Tan, Dingwei Klein, Jacques Bissyand, Tegawend F. Ezzini, Saad |
| contents | Current video generation models excel at creating short, realistic clips, but struggle with longer, multi-scene videos. We introduce \texttt{DreamFactory}, an LLM-based framework that tackles this challenge. \texttt{DreamFactory} leverages multi-agent collaboration principles and a Key Frames Iteration Design Method to ensure consistency and style across long videos. It utilizes Chain of Thought (COT) to address uncertainties inherent in large language models. \texttt{DreamFactory} generates long, stylistically coherent, and complex videos. Evaluating these long-form videos presents a challenge. We propose novel metrics such as Cross-Scene Face Distance Score and Cross-Scene Style Consistency Score. To further research in this area, we contribute the Multi-Scene Videos Dataset containing over 150 human-rated videos. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_11788 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DreamFactory: Pioneering Multi-Scene Long Video Generation with a Multi-Agent Framework Xie, Zhifei Tang, Daniel Tan, Dingwei Klein, Jacques Bissyand, Tegawend F. Ezzini, Saad Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Software Engineering TsingHua University Current video generation models excel at creating short, realistic clips, but struggle with longer, multi-scene videos. We introduce \texttt{DreamFactory}, an LLM-based framework that tackles this challenge. \texttt{DreamFactory} leverages multi-agent collaboration principles and a Key Frames Iteration Design Method to ensure consistency and style across long videos. It utilizes Chain of Thought (COT) to address uncertainties inherent in large language models. \texttt{DreamFactory} generates long, stylistically coherent, and complex videos. Evaluating these long-form videos presents a challenge. We propose novel metrics such as Cross-Scene Face Distance Score and Cross-Scene Style Consistency Score. To further research in this area, we contribute the Multi-Scene Videos Dataset containing over 150 human-rated videos. |
| title | DreamFactory: Pioneering Multi-Scene Long Video Generation with a Multi-Agent Framework |
| topic | Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Software Engineering TsingHua University |
| url | https://arxiv.org/abs/2408.11788 |