Camera Relocalization in Shadow-free Neural Radiance Fields
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929355340382208 |
|---|---|
| author | Xu, Shiyao Liu, Caiyun Chen, Yuantao Zhu, Zhenxin Yan, Zike Shi, Yongliang Zhao, Hao Zhou, Guyue |
| author_facet | Xu, Shiyao Liu, Caiyun Chen, Yuantao Zhu, Zhenxin Yan, Zike Shi, Yongliang Zhao, Hao Zhou, Guyue |
| contents | Camera relocalization is a crucial problem in computer vision and robotics. Recent advancements in neural radiance fields (NeRFs) have shown promise in synthesizing photo-realistic images. Several works have utilized NeRFs for refining camera poses, but they do not account for lighting changes that can affect scene appearance and shadow regions, causing a degraded pose optimization process. In this paper, we propose a two-staged pipeline that normalizes images with varying lighting and shadow conditions to improve camera relocalization. We implement our scene representation upon a hash-encoded NeRF which significantly boosts up the pose optimization process. To account for the noisy image gradient computing problem in grid-based NeRFs, we further propose a re-devised truncated dynamic low-pass filter (TDLF) and a numerical gradient averaging technique to smoothen the process. Experimental results on several datasets with varying lighting conditions demonstrate that our method achieves state-of-the-art results in camera relocalization under varying lighting conditions. Code and data will be made publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_14824 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Camera Relocalization in Shadow-free Neural Radiance Fields Xu, Shiyao Liu, Caiyun Chen, Yuantao Zhu, Zhenxin Yan, Zike Shi, Yongliang Zhao, Hao Zhou, Guyue Computer Vision and Pattern Recognition Robotics Camera relocalization is a crucial problem in computer vision and robotics. Recent advancements in neural radiance fields (NeRFs) have shown promise in synthesizing photo-realistic images. Several works have utilized NeRFs for refining camera poses, but they do not account for lighting changes that can affect scene appearance and shadow regions, causing a degraded pose optimization process. In this paper, we propose a two-staged pipeline that normalizes images with varying lighting and shadow conditions to improve camera relocalization. We implement our scene representation upon a hash-encoded NeRF which significantly boosts up the pose optimization process. To account for the noisy image gradient computing problem in grid-based NeRFs, we further propose a re-devised truncated dynamic low-pass filter (TDLF) and a numerical gradient averaging technique to smoothen the process. Experimental results on several datasets with varying lighting conditions demonstrate that our method achieves state-of-the-art results in camera relocalization under varying lighting conditions. Code and data will be made publicly available. |
| title | Camera Relocalization in Shadow-free Neural Radiance Fields |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2405.14824 |