Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916907101192192 |
|---|---|
| author | Jiang, Zeren Zheng, Chuanxia Laina, Iro Larlus, Diane Vedaldi, Andrea |
| author_facet | Jiang, Zeren Zheng, Chuanxia Laina, Iro Larlus, Diane Vedaldi, Andrea |
| contents | We introduce Geo4D, a method to repurpose video diffusion models for monocular 3D reconstruction of dynamic scenes. By leveraging the strong dynamic priors captured by large-scale pre-trained video models, Geo4D can be trained using only synthetic data while generalizing well to real data in a zero-shot manner. Geo4D predicts several complementary geometric modalities, namely point, disparity, and ray maps. We propose a new multi-modal alignment algorithm to align and fuse these modalities, as well as a sliding window approach at inference time, thus enabling robust and accurate 4D reconstruction of long videos. Extensive experiments across multiple benchmarks show that Geo4D significantly surpasses state-of-the-art video depth estimation methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_07961 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction Jiang, Zeren Zheng, Chuanxia Laina, Iro Larlus, Diane Vedaldi, Andrea Computer Vision and Pattern Recognition I.4.5 We introduce Geo4D, a method to repurpose video diffusion models for monocular 3D reconstruction of dynamic scenes. By leveraging the strong dynamic priors captured by large-scale pre-trained video models, Geo4D can be trained using only synthetic data while generalizing well to real data in a zero-shot manner. Geo4D predicts several complementary geometric modalities, namely point, disparity, and ray maps. We propose a new multi-modal alignment algorithm to align and fuse these modalities, as well as a sliding window approach at inference time, thus enabling robust and accurate 4D reconstruction of long videos. Extensive experiments across multiple benchmarks show that Geo4D significantly surpasses state-of-the-art video depth estimation methods. |
| title | Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction |
| topic | Computer Vision and Pattern Recognition I.4.5 |
| url | https://arxiv.org/abs/2504.07961 |