MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910527691685888 |
|---|---|
| author | Liu, Tianqi Wang, Guangcong Hu, Shoukang Shen, Liao Ye, Xinyi Zang, Yuhang Cao, Zhiguo Li, Wei Liu, Ziwei |
| author_facet | Liu, Tianqi Wang, Guangcong Hu, Shoukang Shen, Liao Ye, Xinyi Zang, Yuhang Cao, Zhiguo Li, Wei Liu, Ziwei |
| contents | We present MVSGaussian, a new generalizable 3D Gaussian representation approach derived from Multi-View Stereo (MVS) that can efficiently reconstruct unseen scenes. Specifically, 1) we leverage MVS to encode geometry-aware Gaussian representations and decode them into Gaussian parameters. 2) To further enhance performance, we propose a hybrid Gaussian rendering that integrates an efficient volume rendering design for novel view synthesis. 3) To support fast fine-tuning for specific scenes, we introduce a multi-view geometric consistent aggregation strategy to effectively aggregate the point clouds generated by the generalizable model, serving as the initialization for per-scene optimization. Compared with previous generalizable NeRF-based methods, which typically require minutes of fine-tuning and seconds of rendering per image, MVSGaussian achieves real-time rendering with better synthesis quality for each scene. Compared with the vanilla 3D-GS, MVSGaussian achieves better view synthesis with less training computational cost. Extensive experiments on DTU, Real Forward-facing, NeRF Synthetic, and Tanks and Temples datasets validate that MVSGaussian attains state-of-the-art performance with convincing generalizability, real-time rendering speed, and fast per-scene optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_12218 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo Liu, Tianqi Wang, Guangcong Hu, Shoukang Shen, Liao Ye, Xinyi Zang, Yuhang Cao, Zhiguo Li, Wei Liu, Ziwei Computer Vision and Pattern Recognition We present MVSGaussian, a new generalizable 3D Gaussian representation approach derived from Multi-View Stereo (MVS) that can efficiently reconstruct unseen scenes. Specifically, 1) we leverage MVS to encode geometry-aware Gaussian representations and decode them into Gaussian parameters. 2) To further enhance performance, we propose a hybrid Gaussian rendering that integrates an efficient volume rendering design for novel view synthesis. 3) To support fast fine-tuning for specific scenes, we introduce a multi-view geometric consistent aggregation strategy to effectively aggregate the point clouds generated by the generalizable model, serving as the initialization for per-scene optimization. Compared with previous generalizable NeRF-based methods, which typically require minutes of fine-tuning and seconds of rendering per image, MVSGaussian achieves real-time rendering with better synthesis quality for each scene. Compared with the vanilla 3D-GS, MVSGaussian achieves better view synthesis with less training computational cost. Extensive experiments on DTU, Real Forward-facing, NeRF Synthetic, and Tanks and Temples datasets validate that MVSGaussian attains state-of-the-art performance with convincing generalizability, real-time rendering speed, and fast per-scene optimization. |
| title | MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.12218 |