Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2304.13445 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913217876328448 |
|---|---|
| author | Sun, Cheng Cai, Guangyan Li, Zhengqin Yan, Kai Zhang, Cheng Marshall, Carl Huang, Jia-Bin Zhao, Shuang Dong, Zhao |
| author_facet | Sun, Cheng Cai, Guangyan Li, Zhengqin Yan, Kai Zhang, Cheng Marshall, Carl Huang, Jia-Bin Zhao, Shuang Dong, Zhao |
| contents | Reconstructing the shape and spatially varying surface appearances of a physical-world object as well as its surrounding illumination based on 2D images (e.g., photographs) of the object has been a long-standing problem in computer vision and graphics. In this paper, we introduce an accurate and highly efficient object reconstruction pipeline combining neural based object reconstruction and physics-based inverse rendering (PBIR). Our pipeline firstly leverages a neural SDF based shape reconstruction to produce high-quality but potentially imperfect object shape. Then, we introduce a neural material and lighting distillation stage to achieve high-quality predictions for material and illumination. In the last stage, initialized by the neural predictions, we perform PBIR to refine the initial results and obtain the final high-quality reconstruction of object shape, material, and illumination. Experimental results demonstrate our pipeline significantly outperforms existing methods quality-wise and performance-wise. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_13445 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Neural-PBIR Reconstruction of Shape, Material, and Illumination Sun, Cheng Cai, Guangyan Li, Zhengqin Yan, Kai Zhang, Cheng Marshall, Carl Huang, Jia-Bin Zhao, Shuang Dong, Zhao Computer Vision and Pattern Recognition Reconstructing the shape and spatially varying surface appearances of a physical-world object as well as its surrounding illumination based on 2D images (e.g., photographs) of the object has been a long-standing problem in computer vision and graphics. In this paper, we introduce an accurate and highly efficient object reconstruction pipeline combining neural based object reconstruction and physics-based inverse rendering (PBIR). Our pipeline firstly leverages a neural SDF based shape reconstruction to produce high-quality but potentially imperfect object shape. Then, we introduce a neural material and lighting distillation stage to achieve high-quality predictions for material and illumination. In the last stage, initialized by the neural predictions, we perform PBIR to refine the initial results and obtain the final high-quality reconstruction of object shape, material, and illumination. Experimental results demonstrate our pipeline significantly outperforms existing methods quality-wise and performance-wise. |
| title | Neural-PBIR Reconstruction of Shape, Material, and Illumination |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2304.13445 |