LAM3D: Large Image-Point-Cloud Alignment Model for 3D Reconstruction from Single Image
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911886249820160 |
|---|---|
| author | Cui, Ruikai Song, Xibin Sun, Weixuan Wang, Senbo Liu, Weizhe Chen, Shenzhou Shang, Taizhang Li, Yang Barnes, Nick Li, Hongdong Ji, Pan |
| author_facet | Cui, Ruikai Song, Xibin Sun, Weixuan Wang, Senbo Liu, Weizhe Chen, Shenzhou Shang, Taizhang Li, Yang Barnes, Nick Li, Hongdong Ji, Pan |
| contents | Large Reconstruction Models have made significant strides in the realm of automated 3D content generation from single or multiple input images. Despite their success, these models often produce 3D meshes with geometric inaccuracies, stemming from the inherent challenges of deducing 3D shapes solely from image data. In this work, we introduce a novel framework, the Large Image and Point Cloud Alignment Model (LAM3D), which utilizes 3D point cloud data to enhance the fidelity of generated 3D meshes. Our methodology begins with the development of a point-cloud-based network that effectively generates precise and meaningful latent tri-planes, laying the groundwork for accurate 3D mesh reconstruction. Building upon this, our Image-Point-Cloud Feature Alignment technique processes a single input image, aligning to the latent tri-planes to imbue image features with robust 3D information. This process not only enriches the image features but also facilitates the production of high-fidelity 3D meshes without the need for multi-view input, significantly reducing geometric distortions. Our approach achieves state-of-the-art high-fidelity 3D mesh reconstruction from a single image in just 6 seconds, and experiments on various datasets demonstrate its effectiveness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_15622 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LAM3D: Large Image-Point-Cloud Alignment Model for 3D Reconstruction from Single Image Cui, Ruikai Song, Xibin Sun, Weixuan Wang, Senbo Liu, Weizhe Chen, Shenzhou Shang, Taizhang Li, Yang Barnes, Nick Li, Hongdong Ji, Pan Computer Vision and Pattern Recognition Large Reconstruction Models have made significant strides in the realm of automated 3D content generation from single or multiple input images. Despite their success, these models often produce 3D meshes with geometric inaccuracies, stemming from the inherent challenges of deducing 3D shapes solely from image data. In this work, we introduce a novel framework, the Large Image and Point Cloud Alignment Model (LAM3D), which utilizes 3D point cloud data to enhance the fidelity of generated 3D meshes. Our methodology begins with the development of a point-cloud-based network that effectively generates precise and meaningful latent tri-planes, laying the groundwork for accurate 3D mesh reconstruction. Building upon this, our Image-Point-Cloud Feature Alignment technique processes a single input image, aligning to the latent tri-planes to imbue image features with robust 3D information. This process not only enriches the image features but also facilitates the production of high-fidelity 3D meshes without the need for multi-view input, significantly reducing geometric distortions. Our approach achieves state-of-the-art high-fidelity 3D mesh reconstruction from a single image in just 6 seconds, and experiments on various datasets demonstrate its effectiveness. |
| title | LAM3D: Large Image-Point-Cloud Alignment Model for 3D Reconstruction from Single Image |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.15622 |