4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913431062315008 |
|---|---|
| author | Wu, Guanjun Yi, Taoran Fang, Jiemin Xie, Lingxi Zhang, Xiaopeng Wei, Wei Liu, Wenyu Tian, Qi Wang, Xinggang |
| author_facet | Wu, Guanjun Yi, Taoran Fang, Jiemin Xie, Lingxi Zhang, Xiaopeng Wei, Wei Liu, Wenyu Tian, Qi Wang, Xinggang |
| contents | Representing and rendering dynamic scenes has been an important but challenging task. Especially, to accurately model complex motions, high efficiency is usually hard to guarantee. To achieve real-time dynamic scene rendering while also enjoying high training and storage efficiency, we propose 4D Gaussian Splatting (4D-GS) as a holistic representation for dynamic scenes rather than applying 3D-GS for each individual frame. In 4D-GS, a novel explicit representation containing both 3D Gaussians and 4D neural voxels is proposed. A decomposed neural voxel encoding algorithm inspired by HexPlane is proposed to efficiently build Gaussian features from 4D neural voxels and then a lightweight MLP is applied to predict Gaussian deformations at novel timestamps. Our 4D-GS method achieves real-time rendering under high resolutions, 82 FPS at an 800$\times$800 resolution on an RTX 3090 GPU while maintaining comparable or better quality than previous state-of-the-art methods. More demos and code are available at https://guanjunwu.github.io/4dgs/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_08528 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering Wu, Guanjun Yi, Taoran Fang, Jiemin Xie, Lingxi Zhang, Xiaopeng Wei, Wei Liu, Wenyu Tian, Qi Wang, Xinggang Computer Vision and Pattern Recognition Graphics Representing and rendering dynamic scenes has been an important but challenging task. Especially, to accurately model complex motions, high efficiency is usually hard to guarantee. To achieve real-time dynamic scene rendering while also enjoying high training and storage efficiency, we propose 4D Gaussian Splatting (4D-GS) as a holistic representation for dynamic scenes rather than applying 3D-GS for each individual frame. In 4D-GS, a novel explicit representation containing both 3D Gaussians and 4D neural voxels is proposed. A decomposed neural voxel encoding algorithm inspired by HexPlane is proposed to efficiently build Gaussian features from 4D neural voxels and then a lightweight MLP is applied to predict Gaussian deformations at novel timestamps. Our 4D-GS method achieves real-time rendering under high resolutions, 82 FPS at an 800$\times$800 resolution on an RTX 3090 GPU while maintaining comparable or better quality than previous state-of-the-art methods. More demos and code are available at https://guanjunwu.github.io/4dgs/. |
| title | 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2310.08528 |