TensoIR: Tensorial Inverse Rendering
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929280166920192 |
|---|---|
| author | Jin, Haian Liu, Isabella Xu, Peijia Zhang, Xiaoshuai Han, Songfang Bi, Sai Zhou, Xiaowei Xu, Zexiang Su, Hao |
| author_facet | Jin, Haian Liu, Isabella Xu, Peijia Zhang, Xiaoshuai Han, Songfang Bi, Sai Zhou, Xiaowei Xu, Zexiang Su, Hao |
| contents | We propose TensoIR, a novel inverse rendering approach based on tensor factorization and neural fields. Unlike previous works that use purely MLP-based neural fields, thus suffering from low capacity and high computation costs, we extend TensoRF, a state-of-the-art approach for radiance field modeling, to estimate scene geometry, surface reflectance, and environment illumination from multi-view images captured under unknown lighting conditions. Our approach jointly achieves radiance field reconstruction and physically-based model estimation, leading to photo-realistic novel view synthesis and relighting results. Benefiting from the efficiency and extensibility of the TensoRF-based representation, our method can accurately model secondary shading effects (like shadows and indirect lighting) and generally support input images captured under single or multiple unknown lighting conditions. The low-rank tensor representation allows us to not only achieve fast and compact reconstruction but also better exploit shared information under an arbitrary number of capturing lighting conditions. We demonstrate the superiority of our method to baseline methods qualitatively and quantitatively on various challenging synthetic and real-world scenes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_12461 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | TensoIR: Tensorial Inverse Rendering Jin, Haian Liu, Isabella Xu, Peijia Zhang, Xiaoshuai Han, Songfang Bi, Sai Zhou, Xiaowei Xu, Zexiang Su, Hao Computer Vision and Pattern Recognition We propose TensoIR, a novel inverse rendering approach based on tensor factorization and neural fields. Unlike previous works that use purely MLP-based neural fields, thus suffering from low capacity and high computation costs, we extend TensoRF, a state-of-the-art approach for radiance field modeling, to estimate scene geometry, surface reflectance, and environment illumination from multi-view images captured under unknown lighting conditions. Our approach jointly achieves radiance field reconstruction and physically-based model estimation, leading to photo-realistic novel view synthesis and relighting results. Benefiting from the efficiency and extensibility of the TensoRF-based representation, our method can accurately model secondary shading effects (like shadows and indirect lighting) and generally support input images captured under single or multiple unknown lighting conditions. The low-rank tensor representation allows us to not only achieve fast and compact reconstruction but also better exploit shared information under an arbitrary number of capturing lighting conditions. We demonstrate the superiority of our method to baseline methods qualitatively and quantitatively on various challenging synthetic and real-world scenes. |
| title | TensoIR: Tensorial Inverse Rendering |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2304.12461 |