Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.03613 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911968084885504 |
|---|---|
| author | Bae, Jeongmin Kim, Seoha Yun, Youngsik Lee, Hahyun Bang, Gun Uh, Youngjung |
| author_facet | Bae, Jeongmin Kim, Seoha Yun, Youngsik Lee, Hahyun Bang, Gun Uh, Youngjung |
| contents | As 3D Gaussian Splatting (3DGS) provides fast and high-quality novel view synthesis, it is a natural extension to deform a canonical 3DGS to multiple frames for representing a dynamic scene. However, previous works fail to accurately reconstruct complex dynamic scenes. We attribute the failure to the design of the deformation field, which is built as a coordinate-based function. This approach is problematic because 3DGS is a mixture of multiple fields centered at the Gaussians, not just a single coordinate-based framework. To resolve this problem, we define the deformation as a function of per-Gaussian embeddings and temporal embeddings. Moreover, we decompose deformations as coarse and fine deformations to model slow and fast movements, respectively. Also, we introduce a local smoothness regularization for per-Gaussian embedding to improve the details in dynamic regions. Project page: https://jeongminb.github.io/e-d3dgs/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_03613 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting Bae, Jeongmin Kim, Seoha Yun, Youngsik Lee, Hahyun Bang, Gun Uh, Youngjung Computer Vision and Pattern Recognition As 3D Gaussian Splatting (3DGS) provides fast and high-quality novel view synthesis, it is a natural extension to deform a canonical 3DGS to multiple frames for representing a dynamic scene. However, previous works fail to accurately reconstruct complex dynamic scenes. We attribute the failure to the design of the deformation field, which is built as a coordinate-based function. This approach is problematic because 3DGS is a mixture of multiple fields centered at the Gaussians, not just a single coordinate-based framework. To resolve this problem, we define the deformation as a function of per-Gaussian embeddings and temporal embeddings. Moreover, we decompose deformations as coarse and fine deformations to model slow and fast movements, respectively. Also, we introduce a local smoothness regularization for per-Gaussian embedding to improve the details in dynamic regions. Project page: https://jeongminb.github.io/e-d3dgs/ |
| title | Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.03613 |