CorrespondentDream: Enhancing 3D Fidelity of Text-to-3D using Cross-View Correspondences
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912030921850880 |
|---|---|
| author | Kim, Seungwook Li, Kejie Deng, Xueqing Shi, Yichun Cho, Minsu Wang, Peng |
| author_facet | Kim, Seungwook Li, Kejie Deng, Xueqing Shi, Yichun Cho, Minsu Wang, Peng |
| contents | Leveraging multi-view diffusion models as priors for 3D optimization have alleviated the problem of 3D consistency, e.g., the Janus face problem or the content drift problem, in zero-shot text-to-3D models. However, the 3D geometric fidelity of the output remains an unresolved issue; albeit the rendered 2D views are realistic, the underlying geometry may contain errors such as unreasonable concavities. In this work, we propose CorrespondentDream, an effective method to leverage annotation-free, cross-view correspondences yielded from the diffusion U-Net to provide additional 3D prior to the NeRF optimization process. We find that these correspondences are strongly consistent with human perception, and by adopting it in our loss design, we are able to produce NeRF models with geometries that are more coherent with common sense, e.g., more smoothed object surface, yielding higher 3D fidelity. We demonstrate the efficacy of our approach through various comparative qualitative results and a solid user study. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_10603 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CorrespondentDream: Enhancing 3D Fidelity of Text-to-3D using Cross-View Correspondences Kim, Seungwook Li, Kejie Deng, Xueqing Shi, Yichun Cho, Minsu Wang, Peng Computer Vision and Pattern Recognition Leveraging multi-view diffusion models as priors for 3D optimization have alleviated the problem of 3D consistency, e.g., the Janus face problem or the content drift problem, in zero-shot text-to-3D models. However, the 3D geometric fidelity of the output remains an unresolved issue; albeit the rendered 2D views are realistic, the underlying geometry may contain errors such as unreasonable concavities. In this work, we propose CorrespondentDream, an effective method to leverage annotation-free, cross-view correspondences yielded from the diffusion U-Net to provide additional 3D prior to the NeRF optimization process. We find that these correspondences are strongly consistent with human perception, and by adopting it in our loss design, we are able to produce NeRF models with geometries that are more coherent with common sense, e.g., more smoothed object surface, yielding higher 3D fidelity. We demonstrate the efficacy of our approach through various comparative qualitative results and a solid user study. |
| title | CorrespondentDream: Enhancing 3D Fidelity of Text-to-3D using Cross-View Correspondences |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.10603 |