Multi-View Large Reconstruction Model via Geometry-Aware Positional Encoding and Attention
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916502006923264 |
|---|---|
| author | Li, Mengfei Long, Xiaoxiao Liang, Yixun Li, Weiyu Liu, Yuan Li, Peng Luo, Wenhan Wang, Wenping Guo, Yike |
| author_facet | Li, Mengfei Long, Xiaoxiao Liang, Yixun Li, Weiyu Liu, Yuan Li, Peng Luo, Wenhan Wang, Wenping Guo, Yike |
| contents | Despite recent advancements in the Large Reconstruction Model (LRM) demonstrating impressive results, when extending its input from single image to multiple images, it exhibits inefficiencies, subpar geometric and texture quality, as well as slower convergence speed than expected. It is attributed to that, LRM formulates 3D reconstruction as a naive images-to-3D translation problem, ignoring the strong 3D coherence among the input images. In this paper, we propose a Multi-view Large Reconstruction Model (M-LRM) designed to reconstruct high-quality 3D shapes from multi-views in a 3D-aware manner. Specifically, we introduce a multi-view consistent cross-attention scheme to enable M-LRM to accurately query information from the input images. Moreover, we employ the 3D priors of the input multi-view images to initialize the triplane tokens. Compared to previous methods, the proposed M-LRM can generate 3D shapes of high fidelity. Experimental studies demonstrate that our model achieves a significant performance gain and faster training convergence. Project page: \url{https://murphylmf.github.io/M-LRM/}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07648 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Multi-View Large Reconstruction Model via Geometry-Aware Positional Encoding and Attention Li, Mengfei Long, Xiaoxiao Liang, Yixun Li, Weiyu Liu, Yuan Li, Peng Luo, Wenhan Wang, Wenping Guo, Yike Computer Vision and Pattern Recognition Despite recent advancements in the Large Reconstruction Model (LRM) demonstrating impressive results, when extending its input from single image to multiple images, it exhibits inefficiencies, subpar geometric and texture quality, as well as slower convergence speed than expected. It is attributed to that, LRM formulates 3D reconstruction as a naive images-to-3D translation problem, ignoring the strong 3D coherence among the input images. In this paper, we propose a Multi-view Large Reconstruction Model (M-LRM) designed to reconstruct high-quality 3D shapes from multi-views in a 3D-aware manner. Specifically, we introduce a multi-view consistent cross-attention scheme to enable M-LRM to accurately query information from the input images. Moreover, we employ the 3D priors of the input multi-view images to initialize the triplane tokens. Compared to previous methods, the proposed M-LRM can generate 3D shapes of high fidelity. Experimental studies demonstrate that our model achieves a significant performance gain and faster training convergence. Project page: \url{https://murphylmf.github.io/M-LRM/}. |
| title | Multi-View Large Reconstruction Model via Geometry-Aware Positional Encoding and Attention |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.07648 |