Deep Geometric Moments Promote Shape Consistency in Text-to-3D Generation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917898680795136 |
|---|---|
| author | Nath, Utkarsh Goel, Rajeev Jeon, Eun Som Kim, Changhoon Min, Kyle Yang, Yezhou Yang, Yingzhen Turaga, Pavan |
| author_facet | Nath, Utkarsh Goel, Rajeev Jeon, Eun Som Kim, Changhoon Min, Kyle Yang, Yezhou Yang, Yingzhen Turaga, Pavan |
| contents | To address the data scarcity associated with 3D assets, 2D-lifting techniques such as Score Distillation Sampling (SDS) have become a widely adopted practice in text-to-3D generation pipelines. However, the diffusion models used in these techniques are prone to viewpoint bias and thus lead to geometric inconsistencies such as the Janus problem. To counter this, we introduce MT3D, a text-to-3D generative model that leverages a high-fidelity 3D object to overcome viewpoint bias and explicitly infuse geometric understanding into the generation pipeline. Firstly, we employ depth maps derived from a high-quality 3D model as control signals to guarantee that the generated 2D images preserve the fundamental shape and structure, thereby reducing the inherent viewpoint bias. Next, we utilize deep geometric moments to ensure geometric consistency in the 3D representation explicitly. By incorporating geometric details from a 3D asset, MT3D enables the creation of diverse and geometrically consistent objects, thereby improving the quality and usability of our 3D representations. Project page and code: https://moment-3d.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_05938 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Deep Geometric Moments Promote Shape Consistency in Text-to-3D Generation Nath, Utkarsh Goel, Rajeev Jeon, Eun Som Kim, Changhoon Min, Kyle Yang, Yezhou Yang, Yingzhen Turaga, Pavan Computer Vision and Pattern Recognition To address the data scarcity associated with 3D assets, 2D-lifting techniques such as Score Distillation Sampling (SDS) have become a widely adopted practice in text-to-3D generation pipelines. However, the diffusion models used in these techniques are prone to viewpoint bias and thus lead to geometric inconsistencies such as the Janus problem. To counter this, we introduce MT3D, a text-to-3D generative model that leverages a high-fidelity 3D object to overcome viewpoint bias and explicitly infuse geometric understanding into the generation pipeline. Firstly, we employ depth maps derived from a high-quality 3D model as control signals to guarantee that the generated 2D images preserve the fundamental shape and structure, thereby reducing the inherent viewpoint bias. Next, we utilize deep geometric moments to ensure geometric consistency in the 3D representation explicitly. By incorporating geometric details from a 3D asset, MT3D enables the creation of diverse and geometrically consistent objects, thereby improving the quality and usability of our 3D representations. Project page and code: https://moment-3d.github.io/ |
| title | Deep Geometric Moments Promote Shape Consistency in Text-to-3D Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.05938 |