MirrorGaussian: Reflecting 3D Gaussians for Reconstructing Mirror Reflections
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929348832919552 |
|---|---|
| author | Liu, Jiayue Tang, Xiao Cheng, Freeman Yang, Roy Li, Zhihao Liu, Jianzhuang Huang, Yi Lin, Jiaqi Liu, Shiyong Wu, Xiaofei Xu, Songcen Yuan, Chun |
| author_facet | Liu, Jiayue Tang, Xiao Cheng, Freeman Yang, Roy Li, Zhihao Liu, Jianzhuang Huang, Yi Lin, Jiaqi Liu, Shiyong Wu, Xiaofei Xu, Songcen Yuan, Chun |
| contents | 3D Gaussian Splatting showcases notable advancements in photo-realistic and real-time novel view synthesis. However, it faces challenges in modeling mirror reflections, which exhibit substantial appearance variations from different viewpoints. To tackle this problem, we present MirrorGaussian, the first method for mirror scene reconstruction with real-time rendering based on 3D Gaussian Splatting. The key insight is grounded on the mirror symmetry between the real-world space and the virtual mirror space. We introduce an intuitive dual-rendering strategy that enables differentiable rasterization of both the real-world 3D Gaussians and the mirrored counterpart obtained by reflecting the former about the mirror plane. All 3D Gaussians are jointly optimized with the mirror plane in an end-to-end framework. MirrorGaussian achieves high-quality and real-time rendering in scenes with mirrors, empowering scene editing like adding new mirrors and objects. Comprehensive experiments on multiple datasets demonstrate that our approach significantly outperforms existing methods, achieving state-of-the-art results. Project page: https://mirror-gaussian.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_11921 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MirrorGaussian: Reflecting 3D Gaussians for Reconstructing Mirror Reflections Liu, Jiayue Tang, Xiao Cheng, Freeman Yang, Roy Li, Zhihao Liu, Jianzhuang Huang, Yi Lin, Jiaqi Liu, Shiyong Wu, Xiaofei Xu, Songcen Yuan, Chun Computer Vision and Pattern Recognition 3D Gaussian Splatting showcases notable advancements in photo-realistic and real-time novel view synthesis. However, it faces challenges in modeling mirror reflections, which exhibit substantial appearance variations from different viewpoints. To tackle this problem, we present MirrorGaussian, the first method for mirror scene reconstruction with real-time rendering based on 3D Gaussian Splatting. The key insight is grounded on the mirror symmetry between the real-world space and the virtual mirror space. We introduce an intuitive dual-rendering strategy that enables differentiable rasterization of both the real-world 3D Gaussians and the mirrored counterpart obtained by reflecting the former about the mirror plane. All 3D Gaussians are jointly optimized with the mirror plane in an end-to-end framework. MirrorGaussian achieves high-quality and real-time rendering in scenes with mirrors, empowering scene editing like adding new mirrors and objects. Comprehensive experiments on multiple datasets demonstrate that our approach significantly outperforms existing methods, achieving state-of-the-art results. Project page: https://mirror-gaussian.github.io/. |
| title | MirrorGaussian: Reflecting 3D Gaussians for Reconstructing Mirror Reflections |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.11921 |