EnvGS: Modeling View-Dependent Appearance with Environment Gaussian
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909554775687168 |
|---|---|
| author | Xie, Tao Chen, Xi Xu, Zhen Xie, Yiman Jin, Yudong Shen, Yujun Peng, Sida Bao, Hujun Zhou, Xiaowei |
| author_facet | Xie, Tao Chen, Xi Xu, Zhen Xie, Yiman Jin, Yudong Shen, Yujun Peng, Sida Bao, Hujun Zhou, Xiaowei |
| contents | Reconstructing complex reflections in real-world scenes from 2D images is essential for achieving photorealistic novel view synthesis. Existing methods that utilize environment maps to model reflections from distant lighting often struggle with high-frequency reflection details and fail to account for near-field reflections. In this work, we introduce EnvGS, a novel approach that employs a set of Gaussian primitives as an explicit 3D representation for capturing reflections of environments. These environment Gaussian primitives are incorporated with base Gaussian primitives to model the appearance of the whole scene. To efficiently render these environment Gaussian primitives, we developed a ray-tracing-based renderer that leverages the GPU's RT core for fast rendering. This allows us to jointly optimize our model for high-quality reconstruction while maintaining real-time rendering speeds. Results from multiple real-world and synthetic datasets demonstrate that our method produces significantly more detailed reflections, achieving the best rendering quality in real-time novel view synthesis. The code is available at https://zju3dv.github.io/envgs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15215 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EnvGS: Modeling View-Dependent Appearance with Environment Gaussian Xie, Tao Chen, Xi Xu, Zhen Xie, Yiman Jin, Yudong Shen, Yujun Peng, Sida Bao, Hujun Zhou, Xiaowei Computer Vision and Pattern Recognition Reconstructing complex reflections in real-world scenes from 2D images is essential for achieving photorealistic novel view synthesis. Existing methods that utilize environment maps to model reflections from distant lighting often struggle with high-frequency reflection details and fail to account for near-field reflections. In this work, we introduce EnvGS, a novel approach that employs a set of Gaussian primitives as an explicit 3D representation for capturing reflections of environments. These environment Gaussian primitives are incorporated with base Gaussian primitives to model the appearance of the whole scene. To efficiently render these environment Gaussian primitives, we developed a ray-tracing-based renderer that leverages the GPU's RT core for fast rendering. This allows us to jointly optimize our model for high-quality reconstruction while maintaining real-time rendering speeds. Results from multiple real-world and synthetic datasets demonstrate that our method produces significantly more detailed reflections, achieving the best rendering quality in real-time novel view synthesis. The code is available at https://zju3dv.github.io/envgs. |
| title | EnvGS: Modeling View-Dependent Appearance with Environment Gaussian |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.15215 |