Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929304610275328 |
|---|---|
| author | Hong, Sunghwan Jung, Jaewoo Shin, Heeseong Yang, Jiaolong Kim, Seungryong Luo, Chong |
| author_facet | Hong, Sunghwan Jung, Jaewoo Shin, Heeseong Yang, Jiaolong Kim, Seungryong Luo, Chong |
| contents | This work delves into the task of pose-free novel view synthesis from stereo pairs, a challenging and pioneering task in 3D vision. Our innovative framework, unlike any before, seamlessly integrates 2D correspondence matching, camera pose estimation, and NeRF rendering, fostering a synergistic enhancement of these tasks. We achieve this through designing an architecture that utilizes a shared representation, which serves as a foundation for enhanced 3D geometry understanding. Capitalizing on the inherent interplay between the tasks, our unified framework is trained end-to-end with the proposed training strategy to improve overall model accuracy. Through extensive evaluations across diverse indoor and outdoor scenes from two real-world datasets, we demonstrate that our approach achieves substantial improvement over previous methodologies, especially in scenarios characterized by extreme viewpoint changes and the absence of accurate camera poses. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_07246 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairs Hong, Sunghwan Jung, Jaewoo Shin, Heeseong Yang, Jiaolong Kim, Seungryong Luo, Chong Computer Vision and Pattern Recognition This work delves into the task of pose-free novel view synthesis from stereo pairs, a challenging and pioneering task in 3D vision. Our innovative framework, unlike any before, seamlessly integrates 2D correspondence matching, camera pose estimation, and NeRF rendering, fostering a synergistic enhancement of these tasks. We achieve this through designing an architecture that utilizes a shared representation, which serves as a foundation for enhanced 3D geometry understanding. Capitalizing on the inherent interplay between the tasks, our unified framework is trained end-to-end with the proposed training strategy to improve overall model accuracy. Through extensive evaluations across diverse indoor and outdoor scenes from two real-world datasets, we demonstrate that our approach achieves substantial improvement over previous methodologies, especially in scenarios characterized by extreme viewpoint changes and the absence of accurate camera poses. |
| title | Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairs |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.07246 |