Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.18525 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912555606212608 |
|---|---|
| author | Han, Haonan Yang, Rui Liao, Huan Xing, Jiankai Xu, Zunnan Yu, Xiaoming Zha, Junwei Li, Xiu Li, Wanhua |
| author_facet | Han, Haonan Yang, Rui Liao, Huan Xing, Jiankai Xu, Zunnan Yu, Xiaoming Zha, Junwei Li, Xiu Li, Wanhua |
| contents | Traditional image-to-3D models often struggle with scenes containing multiple objects due to biases and occlusion complexities. To address this challenge, we present REPARO, a novel approach for compositional 3D asset generation from single images. REPARO employs a two-step process: first, it extracts individual objects from the scene and reconstructs their 3D meshes using off-the-shelf image-to-3D models; then, it optimizes the layout of these meshes through differentiable rendering techniques, ensuring coherent scene composition. By integrating optimal transport-based long-range appearance loss term and high-level semantic loss term in the differentiable rendering, REPARO can effectively recover the layout of 3D assets. The proposed method can significantly enhance object independence, detail accuracy, and overall scene coherence. Extensive evaluation of multi-object scenes demonstrates that our REPARO offers a comprehensive approach to address the complexities of multi-object 3D scene generation from single images. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_18525 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment Han, Haonan Yang, Rui Liao, Huan Xing, Jiankai Xu, Zunnan Yu, Xiaoming Zha, Junwei Li, Xiu Li, Wanhua Computer Vision and Pattern Recognition Traditional image-to-3D models often struggle with scenes containing multiple objects due to biases and occlusion complexities. To address this challenge, we present REPARO, a novel approach for compositional 3D asset generation from single images. REPARO employs a two-step process: first, it extracts individual objects from the scene and reconstructs their 3D meshes using off-the-shelf image-to-3D models; then, it optimizes the layout of these meshes through differentiable rendering techniques, ensuring coherent scene composition. By integrating optimal transport-based long-range appearance loss term and high-level semantic loss term in the differentiable rendering, REPARO can effectively recover the layout of 3D assets. The proposed method can significantly enhance object independence, detail accuracy, and overall scene coherence. Extensive evaluation of multi-object scenes demonstrates that our REPARO offers a comprehensive approach to address the complexities of multi-object 3D scene generation from single images. |
| title | REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.18525 |