Modeling Ambient Scene Dynamics for Free-view Synthesis
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916285868146688 |
|---|---|
| author | Shih, Meng-Li Huang, Jia-Bin Kim, Changil Shah, Rajvi Kopf, Johannes Gao, Chen |
| author_facet | Shih, Meng-Li Huang, Jia-Bin Kim, Changil Shah, Rajvi Kopf, Johannes Gao, Chen |
| contents | We introduce a novel method for dynamic free-view synthesis of an ambient scenes from a monocular capture bringing a immersive quality to the viewing experience. Our method builds upon the recent advancements in 3D Gaussian Splatting (3DGS) that can faithfully reconstruct complex static scenes. Previous attempts to extend 3DGS to represent dynamics have been confined to bounded scenes or require multi-camera captures, and often fail to generalize to unseen motions, limiting their practical application. Our approach overcomes these constraints by leveraging the periodicity of ambient motions to learn the motion trajectory model, coupled with careful regularization. We also propose important practical strategies to improve the visual quality of the baseline 3DGS static reconstructions and to improve memory efficiency critical for GPU-memory intensive learning. We demonstrate high-quality photorealistic novel view synthesis of several ambient natural scenes with intricate textures and fine structural elements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09395 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Modeling Ambient Scene Dynamics for Free-view Synthesis Shih, Meng-Li Huang, Jia-Bin Kim, Changil Shah, Rajvi Kopf, Johannes Gao, Chen Computer Vision and Pattern Recognition We introduce a novel method for dynamic free-view synthesis of an ambient scenes from a monocular capture bringing a immersive quality to the viewing experience. Our method builds upon the recent advancements in 3D Gaussian Splatting (3DGS) that can faithfully reconstruct complex static scenes. Previous attempts to extend 3DGS to represent dynamics have been confined to bounded scenes or require multi-camera captures, and often fail to generalize to unseen motions, limiting their practical application. Our approach overcomes these constraints by leveraging the periodicity of ambient motions to learn the motion trajectory model, coupled with careful regularization. We also propose important practical strategies to improve the visual quality of the baseline 3DGS static reconstructions and to improve memory efficiency critical for GPU-memory intensive learning. We demonstrate high-quality photorealistic novel view synthesis of several ambient natural scenes with intricate textures and fine structural elements. |
| title | Modeling Ambient Scene Dynamics for Free-view Synthesis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.09395 |