MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910669825114112 |
|---|---|
| author | Chen, Yuedong Xu, Haofei Zheng, Chuanxia Zhuang, Bohan Pollefeys, Marc Geiger, Andreas Cham, Tat-Jen Cai, Jianfei |
| author_facet | Chen, Yuedong Xu, Haofei Zheng, Chuanxia Zhuang, Bohan Pollefeys, Marc Geiger, Andreas Cham, Tat-Jen Cai, Jianfei |
| contents | We introduce MVSplat, an efficient model that, given sparse multi-view images as input, predicts clean feed-forward 3D Gaussians. To accurately localize the Gaussian centers, we build a cost volume representation via plane sweeping, where the cross-view feature similarities stored in the cost volume can provide valuable geometry cues to the estimation of depth. We also learn other Gaussian primitives' parameters jointly with the Gaussian centers while only relying on photometric supervision. We demonstrate the importance of the cost volume representation in learning feed-forward Gaussians via extensive experimental evaluations. On the large-scale RealEstate10K and ACID benchmarks, MVSplat achieves state-of-the-art performance with the fastest feed-forward inference speed (22~fps). More impressively, compared to the latest state-of-the-art method pixelSplat, MVSplat uses $10\times$ fewer parameters and infers more than $2\times$ faster while providing higher appearance and geometry quality as well as better cross-dataset generalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_14627 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images Chen, Yuedong Xu, Haofei Zheng, Chuanxia Zhuang, Bohan Pollefeys, Marc Geiger, Andreas Cham, Tat-Jen Cai, Jianfei Computer Vision and Pattern Recognition We introduce MVSplat, an efficient model that, given sparse multi-view images as input, predicts clean feed-forward 3D Gaussians. To accurately localize the Gaussian centers, we build a cost volume representation via plane sweeping, where the cross-view feature similarities stored in the cost volume can provide valuable geometry cues to the estimation of depth. We also learn other Gaussian primitives' parameters jointly with the Gaussian centers while only relying on photometric supervision. We demonstrate the importance of the cost volume representation in learning feed-forward Gaussians via extensive experimental evaluations. On the large-scale RealEstate10K and ACID benchmarks, MVSplat achieves state-of-the-art performance with the fastest feed-forward inference speed (22~fps). More impressively, compared to the latest state-of-the-art method pixelSplat, MVSplat uses $10\times$ fewer parameters and infers more than $2\times$ faster while providing higher appearance and geometry quality as well as better cross-dataset generalization. |
| title | MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.14627 |