NeuS-PIR: Learning Relightable Neural Surface using Pre-Integrated Rendering
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909149628989440 |
|---|---|
| author | Mao, Shi Wu, Chenming Shen, Zhelun Wang, Yifan Wu, Dayan Zhang, Liangjun |
| author_facet | Mao, Shi Wu, Chenming Shen, Zhelun Wang, Yifan Wu, Dayan Zhang, Liangjun |
| contents | This paper presents a method, namely NeuS-PIR, for recovering relightable neural surfaces using pre-integrated rendering from multi-view images or video. Unlike methods based on NeRF and discrete meshes, our method utilizes implicit neural surface representation to reconstruct high-quality geometry, which facilitates the factorization of the radiance field into two components: a spatially varying material field and an all-frequency lighting representation. This factorization, jointly optimized using an adapted differentiable pre-integrated rendering framework with material encoding regularization, in turn addresses the ambiguity of geometry reconstruction and leads to better disentanglement and refinement of each scene property. Additionally, we introduced a method to distil indirect illumination fields from the learned representations, further recovering the complex illumination effect like inter-reflection. Consequently, our method enables advanced applications such as relighting, which can be seamlessly integrated with modern graphics engines. Qualitative and quantitative experiments have shown that NeuS-PIR outperforms existing methods across various tasks on both synthetic and real datasets. Source code is available at https://github.com/Sheldonmao/NeuSPIR |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_07632 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | NeuS-PIR: Learning Relightable Neural Surface using Pre-Integrated Rendering Mao, Shi Wu, Chenming Shen, Zhelun Wang, Yifan Wu, Dayan Zhang, Liangjun Computer Vision and Pattern Recognition Graphics This paper presents a method, namely NeuS-PIR, for recovering relightable neural surfaces using pre-integrated rendering from multi-view images or video. Unlike methods based on NeRF and discrete meshes, our method utilizes implicit neural surface representation to reconstruct high-quality geometry, which facilitates the factorization of the radiance field into two components: a spatially varying material field and an all-frequency lighting representation. This factorization, jointly optimized using an adapted differentiable pre-integrated rendering framework with material encoding regularization, in turn addresses the ambiguity of geometry reconstruction and leads to better disentanglement and refinement of each scene property. Additionally, we introduced a method to distil indirect illumination fields from the learned representations, further recovering the complex illumination effect like inter-reflection. Consequently, our method enables advanced applications such as relighting, which can be seamlessly integrated with modern graphics engines. Qualitative and quantitative experiments have shown that NeuS-PIR outperforms existing methods across various tasks on both synthetic and real datasets. Source code is available at https://github.com/Sheldonmao/NeuSPIR |
| title | NeuS-PIR: Learning Relightable Neural Surface using Pre-Integrated Rendering |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2306.07632 |