DecentNeRFs: Decentralized Neural Radiance Fields from Crowdsourced Images
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916183492526080 |
|---|---|
| author | Tasneem, Zaid Dave, Akshat Singh, Abhishek Tiwary, Kushagra Vepakomma, Praneeth Veeraraghavan, Ashok Raskar, Ramesh |
| author_facet | Tasneem, Zaid Dave, Akshat Singh, Abhishek Tiwary, Kushagra Vepakomma, Praneeth Veeraraghavan, Ashok Raskar, Ramesh |
| contents | Neural radiance fields (NeRFs) show potential for transforming images captured worldwide into immersive 3D visual experiences. However, most of this captured visual data remains siloed in our camera rolls as these images contain personal details. Even if made public, the problem of learning 3D representations of billions of scenes captured daily in a centralized manner is computationally intractable. Our approach, DecentNeRF, is the first attempt at decentralized, crowd-sourced NeRFs that require $\sim 10^4\times$ less server computing for a scene than a centralized approach. Instead of sending the raw data, our approach requires users to send a 3D representation, distributing the high computation cost of training centralized NeRFs between the users. It learns photorealistic scene representations by decomposing users' 3D views into personal and global NeRFs and a novel optimally weighted aggregation of only the latter. We validate the advantage of our approach to learn NeRFs with photorealism and minimal server computation cost on structured synthetic and real-world photo tourism datasets. We further analyze how secure aggregation of global NeRFs in DecentNeRF minimizes the undesired reconstruction of personal content by the server. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_13199 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DecentNeRFs: Decentralized Neural Radiance Fields from Crowdsourced Images Tasneem, Zaid Dave, Akshat Singh, Abhishek Tiwary, Kushagra Vepakomma, Praneeth Veeraraghavan, Ashok Raskar, Ramesh Computer Vision and Pattern Recognition Distributed, Parallel, and Cluster Computing Neural radiance fields (NeRFs) show potential for transforming images captured worldwide into immersive 3D visual experiences. However, most of this captured visual data remains siloed in our camera rolls as these images contain personal details. Even if made public, the problem of learning 3D representations of billions of scenes captured daily in a centralized manner is computationally intractable. Our approach, DecentNeRF, is the first attempt at decentralized, crowd-sourced NeRFs that require $\sim 10^4\times$ less server computing for a scene than a centralized approach. Instead of sending the raw data, our approach requires users to send a 3D representation, distributing the high computation cost of training centralized NeRFs between the users. It learns photorealistic scene representations by decomposing users' 3D views into personal and global NeRFs and a novel optimally weighted aggregation of only the latter. We validate the advantage of our approach to learn NeRFs with photorealism and minimal server computation cost on structured synthetic and real-world photo tourism datasets. We further analyze how secure aggregation of global NeRFs in DecentNeRF minimizes the undesired reconstruction of personal content by the server. |
| title | DecentNeRFs: Decentralized Neural Radiance Fields from Crowdsourced Images |
| topic | Computer Vision and Pattern Recognition Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2403.13199 |