RelitLRM: Generative Relightable Radiance for Large Reconstruction Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914968976228352 |
|---|---|
| author | Zhang, Tianyuan Kuang, Zhengfei Jin, Haian Xu, Zexiang Bi, Sai Tan, Hao Zhang, He Hu, Yiwei Hasan, Milos Freeman, William T. Zhang, Kai Luan, Fujun |
| author_facet | Zhang, Tianyuan Kuang, Zhengfei Jin, Haian Xu, Zexiang Bi, Sai Tan, Hao Zhang, He Hu, Yiwei Hasan, Milos Freeman, William T. Zhang, Kai Luan, Fujun |
| contents | We propose RelitLRM, a Large Reconstruction Model (LRM) for generating high-quality Gaussian splatting representations of 3D objects under novel illuminations from sparse (4-8) posed images captured under unknown static lighting. Unlike prior inverse rendering methods requiring dense captures and slow optimization, often causing artifacts like incorrect highlights or shadow baking, RelitLRM adopts a feed-forward transformer-based model with a novel combination of a geometry reconstructor and a relightable appearance generator based on diffusion. The model is trained end-to-end on synthetic multi-view renderings of objects under varying known illuminations. This architecture design enables to effectively decompose geometry and appearance, resolve the ambiguity between material and lighting, and capture the multi-modal distribution of shadows and specularity in the relit appearance. We show our sparse-view feed-forward RelitLRM offers competitive relighting results to state-of-the-art dense-view optimization-based baselines while being significantly faster. Our project page is available at: https://relit-lrm.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06231 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RelitLRM: Generative Relightable Radiance for Large Reconstruction Models Zhang, Tianyuan Kuang, Zhengfei Jin, Haian Xu, Zexiang Bi, Sai Tan, Hao Zhang, He Hu, Yiwei Hasan, Milos Freeman, William T. Zhang, Kai Luan, Fujun Computer Vision and Pattern Recognition Graphics Machine Learning We propose RelitLRM, a Large Reconstruction Model (LRM) for generating high-quality Gaussian splatting representations of 3D objects under novel illuminations from sparse (4-8) posed images captured under unknown static lighting. Unlike prior inverse rendering methods requiring dense captures and slow optimization, often causing artifacts like incorrect highlights or shadow baking, RelitLRM adopts a feed-forward transformer-based model with a novel combination of a geometry reconstructor and a relightable appearance generator based on diffusion. The model is trained end-to-end on synthetic multi-view renderings of objects under varying known illuminations. This architecture design enables to effectively decompose geometry and appearance, resolve the ambiguity between material and lighting, and capture the multi-modal distribution of shadows and specularity in the relit appearance. We show our sparse-view feed-forward RelitLRM offers competitive relighting results to state-of-the-art dense-view optimization-based baselines while being significantly faster. Our project page is available at: https://relit-lrm.github.io/. |
| title | RelitLRM: Generative Relightable Radiance for Large Reconstruction Models |
| topic | Computer Vision and Pattern Recognition Graphics Machine Learning |
| url | https://arxiv.org/abs/2410.06231 |