Disentangled Generation and Aggregation for Robust Radiance Fields
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913514247946240 |
|---|---|
| author | Shen, Shihe Gao, Huachen Xu, Wangze Peng, Rui Tang, Luyang Xiong, Kaiqiang Jiao, Jianbo Wang, Ronggang |
| author_facet | Shen, Shihe Gao, Huachen Xu, Wangze Peng, Rui Tang, Luyang Xiong, Kaiqiang Jiao, Jianbo Wang, Ronggang |
| contents | The utilization of the triplane-based radiance fields has gained attention in recent years due to its ability to effectively disentangle 3D scenes with a high-quality representation and low computation cost. A key requirement of this method is the precise input of camera poses. However, due to the local update property of the triplane, a similar joint estimation as previous joint pose-NeRF optimization works easily results in local minima. To this end, we propose the Disentangled Triplane Generation module to introduce global feature context and smoothness into triplane learning, which mitigates errors caused by local updating. Then, we propose the Disentangled Plane Aggregation to mitigate the entanglement caused by the common triplane feature aggregation during camera pose updating. In addition, we introduce a two-stage warm-start training strategy to reduce the implicit constraints caused by the triplane generator. Quantitative and qualitative results demonstrate that our proposed method achieves state-of-the-art performance in novel view synthesis with noisy or unknown camera poses, as well as efficient convergence of optimization. Project page: https://gaohchen.github.io/DiGARR/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15715 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Disentangled Generation and Aggregation for Robust Radiance Fields Shen, Shihe Gao, Huachen Xu, Wangze Peng, Rui Tang, Luyang Xiong, Kaiqiang Jiao, Jianbo Wang, Ronggang Computer Vision and Pattern Recognition Graphics The utilization of the triplane-based radiance fields has gained attention in recent years due to its ability to effectively disentangle 3D scenes with a high-quality representation and low computation cost. A key requirement of this method is the precise input of camera poses. However, due to the local update property of the triplane, a similar joint estimation as previous joint pose-NeRF optimization works easily results in local minima. To this end, we propose the Disentangled Triplane Generation module to introduce global feature context and smoothness into triplane learning, which mitigates errors caused by local updating. Then, we propose the Disentangled Plane Aggregation to mitigate the entanglement caused by the common triplane feature aggregation during camera pose updating. In addition, we introduce a two-stage warm-start training strategy to reduce the implicit constraints caused by the triplane generator. Quantitative and qualitative results demonstrate that our proposed method achieves state-of-the-art performance in novel view synthesis with noisy or unknown camera poses, as well as efficient convergence of optimization. Project page: https://gaohchen.github.io/DiGARR/. |
| title | Disentangled Generation and Aggregation for Robust Radiance Fields |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2409.15715 |