DepthSplat: Connecting Gaussian Splatting and Depth
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917967526100992 |
|---|---|
| author | Xu, Haofei Peng, Songyou Wang, Fangjinhua Blum, Hermann Barath, Daniel Geiger, Andreas Pollefeys, Marc |
| author_facet | Xu, Haofei Peng, Songyou Wang, Fangjinhua Blum, Hermann Barath, Daniel Geiger, Andreas Pollefeys, Marc |
| contents | Gaussian splatting and single-view depth estimation are typically studied in isolation. In this paper, we present DepthSplat to connect Gaussian splatting and depth estimation and study their interactions. More specifically, we first contribute a robust multi-view depth model by leveraging pre-trained monocular depth features, leading to high-quality feed-forward 3D Gaussian splatting reconstructions. We also show that Gaussian splatting can serve as an unsupervised pre-training objective for learning powerful depth models from large-scale multi-view posed datasets. We validate the synergy between Gaussian splatting and depth estimation through extensive ablation and cross-task transfer experiments. Our DepthSplat achieves state-of-the-art performance on ScanNet, RealEstate10K and DL3DV datasets in terms of both depth estimation and novel view synthesis, demonstrating the mutual benefits of connecting both tasks. In addition, DepthSplat enables feed-forward reconstruction from 12 input views (512x960 resolutions) in 0.6 seconds. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_13862 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DepthSplat: Connecting Gaussian Splatting and Depth Xu, Haofei Peng, Songyou Wang, Fangjinhua Blum, Hermann Barath, Daniel Geiger, Andreas Pollefeys, Marc Computer Vision and Pattern Recognition Gaussian splatting and single-view depth estimation are typically studied in isolation. In this paper, we present DepthSplat to connect Gaussian splatting and depth estimation and study their interactions. More specifically, we first contribute a robust multi-view depth model by leveraging pre-trained monocular depth features, leading to high-quality feed-forward 3D Gaussian splatting reconstructions. We also show that Gaussian splatting can serve as an unsupervised pre-training objective for learning powerful depth models from large-scale multi-view posed datasets. We validate the synergy between Gaussian splatting and depth estimation through extensive ablation and cross-task transfer experiments. Our DepthSplat achieves state-of-the-art performance on ScanNet, RealEstate10K and DL3DV datasets in terms of both depth estimation and novel view synthesis, demonstrating the mutual benefits of connecting both tasks. In addition, DepthSplat enables feed-forward reconstruction from 12 input views (512x960 resolutions) in 0.6 seconds. |
| title | DepthSplat: Connecting Gaussian Splatting and Depth |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.13862 |