VideoPanda: Video Panoramic Diffusion with Multi-view Attention
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908325515362304 |
|---|---|
| author | Xie, Kevin Sabour, Amirmojtaba Huang, Jiahui Paschalidou, Despoina Klar, Greg Iqbal, Umar Fidler, Sanja Zeng, Xiaohui |
| author_facet | Xie, Kevin Sabour, Amirmojtaba Huang, Jiahui Paschalidou, Despoina Klar, Greg Iqbal, Umar Fidler, Sanja Zeng, Xiaohui |
| contents | High resolution panoramic video content is paramount for immersive experiences in Virtual Reality, but is non-trivial to collect as it requires specialized equipment and intricate camera setups. In this work, we introduce VideoPanda, a novel approach for synthesizing 360$^\circ$ videos conditioned on text or single-view video data. VideoPanda leverages multi-view attention layers to augment a video diffusion model, enabling it to generate consistent multi-view videos that can be combined into immersive panoramic content. VideoPanda is trained jointly using two conditions: text-only and single-view video, and supports autoregressive generation of long-videos. To overcome the computational burden of multi-view video generation, we randomly subsample the duration and camera views used during training and show that the model is able to gracefully generalize to generating more frames during inference. Extensive evaluations on both real-world and synthetic video datasets demonstrate that VideoPanda generates more realistic and coherent 360$^\circ$ panoramas across all input conditions compared to existing methods. Visit the project website at https://research.nvidia.com/labs/toronto-ai/VideoPanda/ for results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_11389 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VideoPanda: Video Panoramic Diffusion with Multi-view Attention Xie, Kevin Sabour, Amirmojtaba Huang, Jiahui Paschalidou, Despoina Klar, Greg Iqbal, Umar Fidler, Sanja Zeng, Xiaohui Graphics Artificial Intelligence Computer Vision and Pattern Recognition High resolution panoramic video content is paramount for immersive experiences in Virtual Reality, but is non-trivial to collect as it requires specialized equipment and intricate camera setups. In this work, we introduce VideoPanda, a novel approach for synthesizing 360$^\circ$ videos conditioned on text or single-view video data. VideoPanda leverages multi-view attention layers to augment a video diffusion model, enabling it to generate consistent multi-view videos that can be combined into immersive panoramic content. VideoPanda is trained jointly using two conditions: text-only and single-view video, and supports autoregressive generation of long-videos. To overcome the computational burden of multi-view video generation, we randomly subsample the duration and camera views used during training and show that the model is able to gracefully generalize to generating more frames during inference. Extensive evaluations on both real-world and synthetic video datasets demonstrate that VideoPanda generates more realistic and coherent 360$^\circ$ panoramas across all input conditions compared to existing methods. Visit the project website at https://research.nvidia.com/labs/toronto-ai/VideoPanda/ for results. |
| title | VideoPanda: Video Panoramic Diffusion with Multi-view Attention |
| topic | Graphics Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.11389 |