ICON: Incremental CONfidence for Joint Pose and Radiance Field Optimization
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909075655098368 |
|---|---|
| author | Wang, Weiyao Gleize, Pierre Tang, Hao Chen, Xingyu Liang, Kevin J Feiszli, Matt |
| author_facet | Wang, Weiyao Gleize, Pierre Tang, Hao Chen, Xingyu Liang, Kevin J Feiszli, Matt |
| contents | Neural Radiance Fields (NeRF) exhibit remarkable performance for Novel View Synthesis (NVS) given a set of 2D images. However, NeRF training requires accurate camera pose for each input view, typically obtained by Structure-from-Motion (SfM) pipelines. Recent works have attempted to relax this constraint, but they still often rely on decent initial poses which they can refine. Here we aim at removing the requirement for pose initialization. We present Incremental CONfidence (ICON), an optimization procedure for training NeRFs from 2D video frames. ICON only assumes smooth camera motion to estimate initial guess for poses. Further, ICON introduces ``confidence": an adaptive measure of model quality used to dynamically reweight gradients. ICON relies on high-confidence poses to learn NeRF, and high-confidence 3D structure (as encoded by NeRF) to learn poses. We show that ICON, without prior pose initialization, achieves superior performance in both CO3D and HO3D versus methods which use SfM pose. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_08937 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ICON: Incremental CONfidence for Joint Pose and Radiance Field Optimization Wang, Weiyao Gleize, Pierre Tang, Hao Chen, Xingyu Liang, Kevin J Feiszli, Matt Computer Vision and Pattern Recognition Neural Radiance Fields (NeRF) exhibit remarkable performance for Novel View Synthesis (NVS) given a set of 2D images. However, NeRF training requires accurate camera pose for each input view, typically obtained by Structure-from-Motion (SfM) pipelines. Recent works have attempted to relax this constraint, but they still often rely on decent initial poses which they can refine. Here we aim at removing the requirement for pose initialization. We present Incremental CONfidence (ICON), an optimization procedure for training NeRFs from 2D video frames. ICON only assumes smooth camera motion to estimate initial guess for poses. Further, ICON introduces ``confidence": an adaptive measure of model quality used to dynamically reweight gradients. ICON relies on high-confidence poses to learn NeRF, and high-confidence 3D structure (as encoded by NeRF) to learn poses. We show that ICON, without prior pose initialization, achieves superior performance in both CO3D and HO3D versus methods which use SfM pose. |
| title | ICON: Incremental CONfidence for Joint Pose and Radiance Field Optimization |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.08937 |