MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priors
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910948768350208 |
|---|---|
| author | Liu, Qingming Liu, Yuan Wang, Jiepeng Lyv, Xianqiang Wang, Peng Wang, Wenping Hou, Junhui |
| author_facet | Liu, Qingming Liu, Yuan Wang, Jiepeng Lyv, Xianqiang Wang, Peng Wang, Wenping Hou, Junhui |
| contents | In this paper, we propose MoDGS, a new pipeline to render novel views of dy namic scenes from a casually captured monocular video. Previous monocular dynamic NeRF or Gaussian Splatting methods strongly rely on the rapid move ment of input cameras to construct multiview consistency but struggle to recon struct dynamic scenes on casually captured input videos whose cameras are either static or move slowly. To address this challenging task, MoDGS adopts recent single-view depth estimation methods to guide the learning of the dynamic scene. Then, a novel 3D-aware initialization method is proposed to learn a reasonable deformation field and a new robust depth loss is proposed to guide the learning of dynamic scene geometry. Comprehensive experiments demonstrate that MoDGS is able to render high-quality novel view images of dynamic scenes from just a casually captured monocular video, which outperforms state-of-the-art meth ods by a significant margin. The code will be publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_00434 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priors Liu, Qingming Liu, Yuan Wang, Jiepeng Lyv, Xianqiang Wang, Peng Wang, Wenping Hou, Junhui Computer Vision and Pattern Recognition In this paper, we propose MoDGS, a new pipeline to render novel views of dy namic scenes from a casually captured monocular video. Previous monocular dynamic NeRF or Gaussian Splatting methods strongly rely on the rapid move ment of input cameras to construct multiview consistency but struggle to recon struct dynamic scenes on casually captured input videos whose cameras are either static or move slowly. To address this challenging task, MoDGS adopts recent single-view depth estimation methods to guide the learning of the dynamic scene. Then, a novel 3D-aware initialization method is proposed to learn a reasonable deformation field and a new robust depth loss is proposed to guide the learning of dynamic scene geometry. Comprehensive experiments demonstrate that MoDGS is able to render high-quality novel view images of dynamic scenes from just a casually captured monocular video, which outperforms state-of-the-art meth ods by a significant margin. The code will be publicly available. |
| title | MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priors |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.00434 |