GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929459622313984 |
|---|---|
| author | Shi, Yahao Wu, Yanmin Wu, Chenming Liu, Xing Zhao, Chen Feng, Haocheng Zhang, Jian Zhou, Bin Ding, Errui Wang, Jingdong |
| author_facet | Shi, Yahao Wu, Yanmin Wu, Chenming Liu, Xing Zhao, Chen Feng, Haocheng Zhang, Jian Zhou, Bin Ding, Errui Wang, Jingdong |
| contents | This paper presents a 3D Gaussian Inverse Rendering (GIR) method, employing 3D Gaussian representations to effectively factorize the scene into material properties, light, and geometry. The key contributions lie in three-fold. We compute the normal of each 3D Gaussian using the shortest eigenvector, with a directional masking scheme forcing accurate normal estimation without external supervision. We adopt an efficient voxel-based indirect illumination tracing scheme that stores direction-aware outgoing radiance in each 3D Gaussian to disentangle secondary illumination for approximating multi-bounce light transport. To further enhance the illumination disentanglement, we represent a high-resolution environmental map with a learnable low-resolution map and a lightweight, fully convolutional network. Our method achieves state-of-the-art performance in both relighting and novel view synthesis tasks among the recently proposed inverse rendering methods while achieving real-time rendering. This substantiates our proposed method's efficacy and broad applicability, highlighting its potential as an influential tool in various real-time interactive graphics applications such as material editing and relighting. The code will be released at https://github.com/guduxiaolang/GIR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_05133 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization Shi, Yahao Wu, Yanmin Wu, Chenming Liu, Xing Zhao, Chen Feng, Haocheng Zhang, Jian Zhou, Bin Ding, Errui Wang, Jingdong Computer Vision and Pattern Recognition This paper presents a 3D Gaussian Inverse Rendering (GIR) method, employing 3D Gaussian representations to effectively factorize the scene into material properties, light, and geometry. The key contributions lie in three-fold. We compute the normal of each 3D Gaussian using the shortest eigenvector, with a directional masking scheme forcing accurate normal estimation without external supervision. We adopt an efficient voxel-based indirect illumination tracing scheme that stores direction-aware outgoing radiance in each 3D Gaussian to disentangle secondary illumination for approximating multi-bounce light transport. To further enhance the illumination disentanglement, we represent a high-resolution environmental map with a learnable low-resolution map and a lightweight, fully convolutional network. Our method achieves state-of-the-art performance in both relighting and novel view synthesis tasks among the recently proposed inverse rendering methods while achieving real-time rendering. This substantiates our proposed method's efficacy and broad applicability, highlighting its potential as an influential tool in various real-time interactive graphics applications such as material editing and relighting. The code will be released at https://github.com/guduxiaolang/GIR. |
| title | GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.05133 |