GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929284979884032 |
|---|---|
| author | Xu, Yinghao Shi, Zifan Yifan, Wang Chen, Hansheng Yang, Ceyuan Peng, Sida Shen, Yujun Wetzstein, Gordon |
| author_facet | Xu, Yinghao Shi, Zifan Yifan, Wang Chen, Hansheng Yang, Ceyuan Peng, Sida Shen, Yujun Wetzstein, Gordon |
| contents | We introduce GRM, a large-scale reconstructor capable of recovering a 3D asset from sparse-view images in around 0.1s. GRM is a feed-forward transformer-based model that efficiently incorporates multi-view information to translate the input pixels into pixel-aligned Gaussians, which are unprojected to create a set of densely distributed 3D Gaussians representing a scene. Together, our transformer architecture and the use of 3D Gaussians unlock a scalable and efficient reconstruction framework. Extensive experimental results demonstrate the superiority of our method over alternatives regarding both reconstruction quality and efficiency. We also showcase the potential of GRM in generative tasks, i.e., text-to-3D and image-to-3D, by integrating it with existing multi-view diffusion models. Our project website is at: https://justimyhxu.github.io/projects/grm/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_14621 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation Xu, Yinghao Shi, Zifan Yifan, Wang Chen, Hansheng Yang, Ceyuan Peng, Sida Shen, Yujun Wetzstein, Gordon Computer Vision and Pattern Recognition We introduce GRM, a large-scale reconstructor capable of recovering a 3D asset from sparse-view images in around 0.1s. GRM is a feed-forward transformer-based model that efficiently incorporates multi-view information to translate the input pixels into pixel-aligned Gaussians, which are unprojected to create a set of densely distributed 3D Gaussians representing a scene. Together, our transformer architecture and the use of 3D Gaussians unlock a scalable and efficient reconstruction framework. Extensive experimental results demonstrate the superiority of our method over alternatives regarding both reconstruction quality and efficiency. We also showcase the potential of GRM in generative tasks, i.e., text-to-3D and image-to-3D, by integrating it with existing multi-view diffusion models. Our project website is at: https://justimyhxu.github.io/projects/grm/. |
| title | GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.14621 |