Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2508.15720 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916911453831168 |
|---|---|
| author | Liu, Zhiheng Deng, Xueqing Chen, Shoufa Wang, Angtian Guo, Qiushan Han, Mingfei Xue, Zeyue Chen, Mengzhao Luo, Ping Yang, Linjie |
| author_facet | Liu, Zhiheng Deng, Xueqing Chen, Shoufa Wang, Angtian Guo, Qiushan Han, Mingfei Xue, Zeyue Chen, Mengzhao Luo, Ping Yang, Linjie |
| contents | Generative video modeling has made significant strides, yet ensuring structural and temporal consistency over long sequences remains a challenge. Current methods predominantly rely on RGB signals, leading to accumulated errors in object structure and motion over extended durations. To address these issues, we introduce WorldWeaver, a robust framework for long video generation that jointly models RGB frames and perceptual conditions within a unified long-horizon modeling scheme. Our training framework offers three key advantages. First, by jointly predicting perceptual conditions and color information from a unified representation, it significantly enhances temporal consistency and motion dynamics. Second, by leveraging depth cues, which we observe to be more resistant to drift than RGB, we construct a memory bank that preserves clearer contextual information, improving quality in long-horizon video generation. Third, we employ segmented noise scheduling for training prediction groups, which further mitigates drift and reduces computational cost. Extensive experiments on both diffusion- and rectified flow-based models demonstrate the effectiveness of WorldWeaver in reducing temporal drift and improving the fidelity of generated videos. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_15720 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception Liu, Zhiheng Deng, Xueqing Chen, Shoufa Wang, Angtian Guo, Qiushan Han, Mingfei Xue, Zeyue Chen, Mengzhao Luo, Ping Yang, Linjie Computer Vision and Pattern Recognition Generative video modeling has made significant strides, yet ensuring structural and temporal consistency over long sequences remains a challenge. Current methods predominantly rely on RGB signals, leading to accumulated errors in object structure and motion over extended durations. To address these issues, we introduce WorldWeaver, a robust framework for long video generation that jointly models RGB frames and perceptual conditions within a unified long-horizon modeling scheme. Our training framework offers three key advantages. First, by jointly predicting perceptual conditions and color information from a unified representation, it significantly enhances temporal consistency and motion dynamics. Second, by leveraging depth cues, which we observe to be more resistant to drift than RGB, we construct a memory bank that preserves clearer contextual information, improving quality in long-horizon video generation. Third, we employ segmented noise scheduling for training prediction groups, which further mitigates drift and reduces computational cost. Extensive experiments on both diffusion- and rectified flow-based models demonstrate the effectiveness of WorldWeaver in reducing temporal drift and improving the fidelity of generated videos. |
| title | WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.15720 |