SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916661425078272 |
|---|---|
| author | Fan, Cheng-De Chang, Chen-Wei Liu, Yi-Ruei Lee, Jie-Ying Huang, Jiun-Long Tseng, Yu-Chee Liu, Yu-Lun |
| author_facet | Fan, Cheng-De Chang, Chen-Wei Liu, Yi-Ruei Lee, Jie-Ying Huang, Jiun-Long Tseng, Yu-Chee Liu, Yu-Lun |
| contents | We present SpectroMotion, a novel approach that combines 3D Gaussian Splatting (3DGS) with physically-based rendering (PBR) and deformation fields to reconstruct dynamic specular scenes. Previous methods extending 3DGS to model dynamic scenes have struggled to represent specular surfaces accurately. Our method addresses this limitation by introducing a residual correction technique for accurate surface normal computation during deformation, complemented by a deformable environment map that adapts to time-varying lighting conditions. We implement a coarse-to-fine training strategy significantly enhancing scene geometry and specular color prediction. It is the only existing 3DGS method capable of synthesizing photorealistic real-world dynamic specular scenes, outperforming state-of-the-art methods in rendering complex, dynamic, and specular scenes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_17249 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes Fan, Cheng-De Chang, Chen-Wei Liu, Yi-Ruei Lee, Jie-Ying Huang, Jiun-Long Tseng, Yu-Chee Liu, Yu-Lun Computer Vision and Pattern Recognition We present SpectroMotion, a novel approach that combines 3D Gaussian Splatting (3DGS) with physically-based rendering (PBR) and deformation fields to reconstruct dynamic specular scenes. Previous methods extending 3DGS to model dynamic scenes have struggled to represent specular surfaces accurately. Our method addresses this limitation by introducing a residual correction technique for accurate surface normal computation during deformation, complemented by a deformable environment map that adapts to time-varying lighting conditions. We implement a coarse-to-fine training strategy significantly enhancing scene geometry and specular color prediction. It is the only existing 3DGS method capable of synthesizing photorealistic real-world dynamic specular scenes, outperforming state-of-the-art methods in rendering complex, dynamic, and specular scenes. |
| title | SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.17249 |