Incorporating dense metric depth into neural 3D representations for view synthesis and relighting
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916382815289344 |
|---|---|
| author | Chaudhury, Arkadeep Narayan Vasiljevic, Igor Zakharov, Sergey Guizilini, Vitor Ambrus, Rares Narasimhan, Srinivasa Atkeson, Christopher G. |
| author_facet | Chaudhury, Arkadeep Narayan Vasiljevic, Igor Zakharov, Sergey Guizilini, Vitor Ambrus, Rares Narasimhan, Srinivasa Atkeson, Christopher G. |
| contents | Synthesizing accurate geometry and photo-realistic appearance of small scenes is an active area of research with compelling use cases in gaming, virtual reality, robotic-manipulation, autonomous driving, convenient product capture, and consumer-level photography. When applying scene geometry and appearance estimation techniques to robotics, we found that the narrow cone of possible viewpoints due to the limited range of robot motion and scene clutter caused current estimation techniques to produce poor quality estimates or even fail. On the other hand, in robotic applications, dense metric depth can often be measured directly using stereo and illumination can be controlled. Depth can provide a good initial estimate of the object geometry to improve reconstruction, while multi-illumination images can facilitate relighting. In this work we demonstrate a method to incorporate dense metric depth into the training of neural 3D representations and address an artifact observed while jointly refining geometry and appearance by disambiguating between texture and geometry edges. We also discuss a multi-flash stereo camera system developed to capture the necessary data for our pipeline and show results on relighting and view synthesis with a few training views. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_03061 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Incorporating dense metric depth into neural 3D representations for view synthesis and relighting Chaudhury, Arkadeep Narayan Vasiljevic, Igor Zakharov, Sergey Guizilini, Vitor Ambrus, Rares Narasimhan, Srinivasa Atkeson, Christopher G. Computer Vision and Pattern Recognition Graphics Robotics Synthesizing accurate geometry and photo-realistic appearance of small scenes is an active area of research with compelling use cases in gaming, virtual reality, robotic-manipulation, autonomous driving, convenient product capture, and consumer-level photography. When applying scene geometry and appearance estimation techniques to robotics, we found that the narrow cone of possible viewpoints due to the limited range of robot motion and scene clutter caused current estimation techniques to produce poor quality estimates or even fail. On the other hand, in robotic applications, dense metric depth can often be measured directly using stereo and illumination can be controlled. Depth can provide a good initial estimate of the object geometry to improve reconstruction, while multi-illumination images can facilitate relighting. In this work we demonstrate a method to incorporate dense metric depth into the training of neural 3D representations and address an artifact observed while jointly refining geometry and appearance by disambiguating between texture and geometry edges. We also discuss a multi-flash stereo camera system developed to capture the necessary data for our pipeline and show results on relighting and view synthesis with a few training views. |
| title | Incorporating dense metric depth into neural 3D representations for view synthesis and relighting |
| topic | Computer Vision and Pattern Recognition Graphics Robotics |
| url | https://arxiv.org/abs/2409.03061 |