Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.07371 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916516572692480 |
|---|---|
| author | Ge, Wenhang Lin, Jiantao Shen, Guibao Feng, Jiawei Hu, Tao Xu, Xinli Chen, Ying-Cong |
| author_facet | Ge, Wenhang Lin, Jiantao Shen, Guibao Feng, Jiawei Hu, Tao Xu, Xinli Chen, Ying-Cong |
| contents | We propose PRM, a novel photometric stereo based large reconstruction model to reconstruct high-quality meshes with fine-grained local details. Unlike previous large reconstruction models that prepare images under fixed and simple lighting as both input and supervision, PRM renders photometric stereo images by varying materials and lighting for the purposes, which not only improves the precise local details by providing rich photometric cues but also increases the model robustness to variations in the appearance of input images. To offer enhanced flexibility of images rendering, we incorporate a real-time physically-based rendering (PBR) method and mesh rasterization for online images rendering. Moreover, in employing an explicit mesh as our 3D representation, PRM ensures the application of differentiable PBR, which supports the utilization of multiple photometric supervisions and better models the specular color for high-quality geometry optimization. Our PRM leverages photometric stereo images to achieve high-quality reconstructions with fine-grained local details, even amidst sophisticated image appearances. Extensive experiments demonstrate that PRM significantly outperforms other models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_07371 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PRM: Photometric Stereo based Large Reconstruction Model Ge, Wenhang Lin, Jiantao Shen, Guibao Feng, Jiawei Hu, Tao Xu, Xinli Chen, Ying-Cong Computer Vision and Pattern Recognition Graphics We propose PRM, a novel photometric stereo based large reconstruction model to reconstruct high-quality meshes with fine-grained local details. Unlike previous large reconstruction models that prepare images under fixed and simple lighting as both input and supervision, PRM renders photometric stereo images by varying materials and lighting for the purposes, which not only improves the precise local details by providing rich photometric cues but also increases the model robustness to variations in the appearance of input images. To offer enhanced flexibility of images rendering, we incorporate a real-time physically-based rendering (PBR) method and mesh rasterization for online images rendering. Moreover, in employing an explicit mesh as our 3D representation, PRM ensures the application of differentiable PBR, which supports the utilization of multiple photometric supervisions and better models the specular color for high-quality geometry optimization. Our PRM leverages photometric stereo images to achieve high-quality reconstructions with fine-grained local details, even amidst sophisticated image appearances. Extensive experiments demonstrate that PRM significantly outperforms other models. |
| title | PRM: Photometric Stereo based Large Reconstruction Model |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2412.07371 |