PanoDreamer: Optimization-Based Single Image to 360 3D Scene With Diffusion
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912836033183744 |
|---|---|
| author | Paliwal, Avinash Zhou, Xilong Tsarov, Andrii Kalantari, Nima Khademi |
| author_facet | Paliwal, Avinash Zhou, Xilong Tsarov, Andrii Kalantari, Nima Khademi |
| contents | In this paper, we present PanoDreamer, a novel method for producing a coherent 360° 3D scene from a single input image. Unlike existing methods that generate the scene sequentially, we frame the problem as single-image panorama and depth estimation. Once the coherent panoramic image and its corresponding depth are obtained, the scene can be reconstructed by inpainting the small occluded regions and projecting them into 3D space. Our key contribution is formulating single-image panorama and depth estimation as two optimization tasks and introducing alternating minimization strategies to effectively solve their objectives. We demonstrate that our approach outperforms existing techniques in single-image 360° 3D scene reconstruction in terms of consistency and overall quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04827 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PanoDreamer: Optimization-Based Single Image to 360 3D Scene With Diffusion Paliwal, Avinash Zhou, Xilong Tsarov, Andrii Kalantari, Nima Khademi Computer Vision and Pattern Recognition Graphics In this paper, we present PanoDreamer, a novel method for producing a coherent 360° 3D scene from a single input image. Unlike existing methods that generate the scene sequentially, we frame the problem as single-image panorama and depth estimation. Once the coherent panoramic image and its corresponding depth are obtained, the scene can be reconstructed by inpainting the small occluded regions and projecting them into 3D space. Our key contribution is formulating single-image panorama and depth estimation as two optimization tasks and introducing alternating minimization strategies to effectively solve their objectives. We demonstrate that our approach outperforms existing techniques in single-image 360° 3D scene reconstruction in terms of consistency and overall quality. |
| title | PanoDreamer: Optimization-Based Single Image to 360 3D Scene With Diffusion |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2412.04827 |