DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912366855192576 |
|---|---|
| author | Zhao, Qitao Lin, Amy Tan, Jeff Zhang, Jason Y. Ramanan, Deva Tulsiani, Shubham |
| author_facet | Zhao, Qitao Lin, Amy Tan, Jeff Zhang, Jason Y. Ramanan, Deva Tulsiani, Shubham |
| contents | Current Structure-from-Motion (SfM) methods typically follow a two-stage pipeline, combining learned or geometric pairwise reasoning with a subsequent global optimization step. In contrast, we propose a data-driven multi-view reasoning approach that directly infers 3D scene geometry and camera poses from multi-view images. Our framework, DiffusionSfM, parameterizes scene geometry and cameras as pixel-wise ray origins and endpoints in a global frame and employs a transformer-based denoising diffusion model to predict them from multi-view inputs. To address practical challenges in training diffusion models with missing data and unbounded scene coordinates, we introduce specialized mechanisms that ensure robust learning. We empirically validate DiffusionSfM on both synthetic and real datasets, demonstrating that it outperforms classical and learning-based approaches while naturally modeling uncertainty. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_05473 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion Zhao, Qitao Lin, Amy Tan, Jeff Zhang, Jason Y. Ramanan, Deva Tulsiani, Shubham Computer Vision and Pattern Recognition Current Structure-from-Motion (SfM) methods typically follow a two-stage pipeline, combining learned or geometric pairwise reasoning with a subsequent global optimization step. In contrast, we propose a data-driven multi-view reasoning approach that directly infers 3D scene geometry and camera poses from multi-view images. Our framework, DiffusionSfM, parameterizes scene geometry and cameras as pixel-wise ray origins and endpoints in a global frame and employs a transformer-based denoising diffusion model to predict them from multi-view inputs. To address practical challenges in training diffusion models with missing data and unbounded scene coordinates, we introduce specialized mechanisms that ensure robust learning. We empirically validate DiffusionSfM on both synthetic and real datasets, demonstrating that it outperforms classical and learning-based approaches while naturally modeling uncertainty. |
| title | DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.05473 |