CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913617913315328 |
|---|---|
| author | Wu, Rundi Gao, Ruiqi Poole, Ben Trevithick, Alex Zheng, Changxi Barron, Jonathan T. Holynski, Aleksander |
| author_facet | Wu, Rundi Gao, Ruiqi Poole, Ben Trevithick, Alex Zheng, Changxi Barron, Jonathan T. Holynski, Aleksander |
| contents | We present CAT4D, a method for creating 4D (dynamic 3D) scenes from monocular video. CAT4D leverages a multi-view video diffusion model trained on a diverse combination of datasets to enable novel view synthesis at any specified camera poses and timestamps. Combined with a novel sampling approach, this model can transform a single monocular video into a multi-view video, enabling robust 4D reconstruction via optimization of a deformable 3D Gaussian representation. We demonstrate competitive performance on novel view synthesis and dynamic scene reconstruction benchmarks, and highlight the creative capabilities for 4D scene generation from real or generated videos. See our project page for results and interactive demos: https://cat-4d.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_18613 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models Wu, Rundi Gao, Ruiqi Poole, Ben Trevithick, Alex Zheng, Changxi Barron, Jonathan T. Holynski, Aleksander Computer Vision and Pattern Recognition We present CAT4D, a method for creating 4D (dynamic 3D) scenes from monocular video. CAT4D leverages a multi-view video diffusion model trained on a diverse combination of datasets to enable novel view synthesis at any specified camera poses and timestamps. Combined with a novel sampling approach, this model can transform a single monocular video into a multi-view video, enabling robust 4D reconstruction via optimization of a deformable 3D Gaussian representation. We demonstrate competitive performance on novel view synthesis and dynamic scene reconstruction benchmarks, and highlight the creative capabilities for 4D scene generation from real or generated videos. See our project page for results and interactive demos: https://cat-4d.github.io/. |
| title | CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.18613 |