Materialist: Physically Based Editing Using Single-Image Inverse Rendering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918495038472192 |
|---|---|
| author | Wang, Lezhong Tran, Duc Minh Cui, Ruiqi TG, Thomson Dahl, Anders Bjorholm Bigdeli, Siavash Arjomand Frisvad, Jeppe Revall Chandraker, Manmohan |
| author_facet | Wang, Lezhong Tran, Duc Minh Cui, Ruiqi TG, Thomson Dahl, Anders Bjorholm Bigdeli, Siavash Arjomand Frisvad, Jeppe Revall Chandraker, Manmohan |
| contents | Achieving physically consistent image editing remains a significant challenge in computer vision. Existing image editing methods typically rely on neural networks, which struggle to accurately handle shadows and refractions. Conversely, physics-based inverse rendering often requires multi-view optimization, limiting its practicality in single-image scenarios. In this paper, we propose Materialist, a neural-initialized physically based rendering pipeline for single-image inverse rendering. Unlike previous hybrid methods that use physics to guide neural generation, our method leverages neural networks to predict initial material properties, which are then rigorously optimized via progressive differentiable rendering. Our approach enables a range of applications, including material editing, object insertion, and relighting, while also introducing an effective method for editing material transparency via ray-traced refraction without requiring full scene geometry. Furthermore, our envmap estimation method also achieves competitive performance, further enhancing the accuracy of image editing task. Experiments demonstrate strong performance across synthetic and real-world datasets, excelling even on challenging out-of-domain images. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_03717 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Materialist: Physically Based Editing Using Single-Image Inverse Rendering Wang, Lezhong Tran, Duc Minh Cui, Ruiqi TG, Thomson Dahl, Anders Bjorholm Bigdeli, Siavash Arjomand Frisvad, Jeppe Revall Chandraker, Manmohan Computer Vision and Pattern Recognition Artificial Intelligence Graphics Achieving physically consistent image editing remains a significant challenge in computer vision. Existing image editing methods typically rely on neural networks, which struggle to accurately handle shadows and refractions. Conversely, physics-based inverse rendering often requires multi-view optimization, limiting its practicality in single-image scenarios. In this paper, we propose Materialist, a neural-initialized physically based rendering pipeline for single-image inverse rendering. Unlike previous hybrid methods that use physics to guide neural generation, our method leverages neural networks to predict initial material properties, which are then rigorously optimized via progressive differentiable rendering. Our approach enables a range of applications, including material editing, object insertion, and relighting, while also introducing an effective method for editing material transparency via ray-traced refraction without requiring full scene geometry. Furthermore, our envmap estimation method also achieves competitive performance, further enhancing the accuracy of image editing task. Experiments demonstrate strong performance across synthetic and real-world datasets, excelling even on challenging out-of-domain images. |
| title | Materialist: Physically Based Editing Using Single-Image Inverse Rendering |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Graphics |
| url | https://arxiv.org/abs/2501.03717 |