Manifold Sampling for Differentiable Uncertainty in Radiance Fields
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916401030103040 |
|---|---|
| author | Lyu, Linjie Tewari, Ayush Habermann, Marc Saito, Shunsuke Zollhöfer, Michael Leimkühler, Thomas Theobalt, Christian |
| author_facet | Lyu, Linjie Tewari, Ayush Habermann, Marc Saito, Shunsuke Zollhöfer, Michael Leimkühler, Thomas Theobalt, Christian |
| contents | Radiance fields are powerful and, hence, popular models for representing the appearance of complex scenes. Yet, constructing them based on image observations gives rise to ambiguities and uncertainties. We propose a versatile approach for learning Gaussian radiance fields with explicit and fine-grained uncertainty estimates that impose only little additional cost compared to uncertainty-agnostic training. Our key observation is that uncertainties can be modeled as a low-dimensional manifold in the space of radiance field parameters that is highly amenable to Monte Carlo sampling. Importantly, our uncertainties are differentiable and, thus, allow for gradient-based optimization of subsequent captures that optimally reduce ambiguities. We demonstrate state-of-the-art performance on next-best-view planning tasks, including high-dimensional illumination planning for optimal radiance field relighting quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_12661 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Manifold Sampling for Differentiable Uncertainty in Radiance Fields Lyu, Linjie Tewari, Ayush Habermann, Marc Saito, Shunsuke Zollhöfer, Michael Leimkühler, Thomas Theobalt, Christian Computer Vision and Pattern Recognition Graphics Radiance fields are powerful and, hence, popular models for representing the appearance of complex scenes. Yet, constructing them based on image observations gives rise to ambiguities and uncertainties. We propose a versatile approach for learning Gaussian radiance fields with explicit and fine-grained uncertainty estimates that impose only little additional cost compared to uncertainty-agnostic training. Our key observation is that uncertainties can be modeled as a low-dimensional manifold in the space of radiance field parameters that is highly amenable to Monte Carlo sampling. Importantly, our uncertainties are differentiable and, thus, allow for gradient-based optimization of subsequent captures that optimally reduce ambiguities. We demonstrate state-of-the-art performance on next-best-view planning tasks, including high-dimensional illumination planning for optimal radiance field relighting quality. |
| title | Manifold Sampling for Differentiable Uncertainty in Radiance Fields |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2409.12661 |