DynaMoN: Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917869143457792 |
|---|---|
| author | Schischka, Nicolas Schieber, Hannah Karaoglu, Mert Asim Görgülü, Melih Grötzner, Florian Ladikos, Alexander Roth, Daniel Navab, Nassir Busam, Benjamin |
| author_facet | Schischka, Nicolas Schieber, Hannah Karaoglu, Mert Asim Görgülü, Melih Grötzner, Florian Ladikos, Alexander Roth, Daniel Navab, Nassir Busam, Benjamin |
| contents | The accurate reconstruction of dynamic scenes with neural radiance fields is significantly dependent on the estimation of camera poses. Widely used structure-from-motion pipelines encounter difficulties in accurately tracking the camera trajectory when faced with separate dynamics of the scene content and the camera movement. To address this challenge, we propose Dynamic Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields (DynaMoN). DynaMoN utilizes semantic segmentation and generic motion masks to handle dynamic content for initial camera pose estimation and statics-focused ray sampling for fast and accurate novel-view synthesis. Our novel iterative learning scheme switches between training the NeRF and updating the pose parameters for an improved reconstruction and trajectory estimation quality. The proposed pipeline shows significant acceleration of the training process. We extensively evaluate our approach on two real-world dynamic datasets, the TUM RGB-D dataset and the BONN RGB-D Dynamic dataset. DynaMoN improves over the state-of-the-art both in terms of reconstruction quality and trajectory accuracy. We plan to make our code public to enhance research in this area. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_08927 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | DynaMoN: Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields Schischka, Nicolas Schieber, Hannah Karaoglu, Mert Asim Görgülü, Melih Grötzner, Florian Ladikos, Alexander Roth, Daniel Navab, Nassir Busam, Benjamin Computer Vision and Pattern Recognition The accurate reconstruction of dynamic scenes with neural radiance fields is significantly dependent on the estimation of camera poses. Widely used structure-from-motion pipelines encounter difficulties in accurately tracking the camera trajectory when faced with separate dynamics of the scene content and the camera movement. To address this challenge, we propose Dynamic Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields (DynaMoN). DynaMoN utilizes semantic segmentation and generic motion masks to handle dynamic content for initial camera pose estimation and statics-focused ray sampling for fast and accurate novel-view synthesis. Our novel iterative learning scheme switches between training the NeRF and updating the pose parameters for an improved reconstruction and trajectory estimation quality. The proposed pipeline shows significant acceleration of the training process. We extensively evaluate our approach on two real-world dynamic datasets, the TUM RGB-D dataset and the BONN RGB-D Dynamic dataset. DynaMoN improves over the state-of-the-art both in terms of reconstruction quality and trajectory accuracy. We plan to make our code public to enhance research in this area. |
| title | DynaMoN: Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2309.08927 |