DreamDissector: Learning Disentangled Text-to-3D Generation from 2D Diffusion Priors
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909265388634112 |
|---|---|
| author | Yan, Zizheng Zhou, Jiapeng Meng, Fanpeng Wu, Yushuang Qiu, Lingteng Ye, Zisheng Cui, Shuguang Chen, Guanying Han, Xiaoguang |
| author_facet | Yan, Zizheng Zhou, Jiapeng Meng, Fanpeng Wu, Yushuang Qiu, Lingteng Ye, Zisheng Cui, Shuguang Chen, Guanying Han, Xiaoguang |
| contents | Text-to-3D generation has recently seen significant progress. To enhance its practicality in real-world applications, it is crucial to generate multiple independent objects with interactions, similar to layer-compositing in 2D image editing. However, existing text-to-3D methods struggle with this task, as they are designed to generate either non-independent objects or independent objects lacking spatially plausible interactions. Addressing this, we propose DreamDissector, a text-to-3D method capable of generating multiple independent objects with interactions. DreamDissector accepts a multi-object text-to-3D NeRF as input and produces independent textured meshes. To achieve this, we introduce the Neural Category Field (NeCF) for disentangling the input NeRF. Additionally, we present the Category Score Distillation Sampling (CSDS), facilitated by a Deep Concept Mining (DCM) module, to tackle the concept gap issue in diffusion models. By leveraging NeCF and CSDS, we can effectively derive sub-NeRFs from the original scene. Further refinement enhances geometry and texture. Our experimental results validate the effectiveness of DreamDissector, providing users with novel means to control 3D synthesis at the object level and potentially opening avenues for various creative applications in the future. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_16260 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DreamDissector: Learning Disentangled Text-to-3D Generation from 2D Diffusion Priors Yan, Zizheng Zhou, Jiapeng Meng, Fanpeng Wu, Yushuang Qiu, Lingteng Ye, Zisheng Cui, Shuguang Chen, Guanying Han, Xiaoguang Computer Vision and Pattern Recognition Text-to-3D generation has recently seen significant progress. To enhance its practicality in real-world applications, it is crucial to generate multiple independent objects with interactions, similar to layer-compositing in 2D image editing. However, existing text-to-3D methods struggle with this task, as they are designed to generate either non-independent objects or independent objects lacking spatially plausible interactions. Addressing this, we propose DreamDissector, a text-to-3D method capable of generating multiple independent objects with interactions. DreamDissector accepts a multi-object text-to-3D NeRF as input and produces independent textured meshes. To achieve this, we introduce the Neural Category Field (NeCF) for disentangling the input NeRF. Additionally, we present the Category Score Distillation Sampling (CSDS), facilitated by a Deep Concept Mining (DCM) module, to tackle the concept gap issue in diffusion models. By leveraging NeCF and CSDS, we can effectively derive sub-NeRFs from the original scene. Further refinement enhances geometry and texture. Our experimental results validate the effectiveness of DreamDissector, providing users with novel means to control 3D synthesis at the object level and potentially opening avenues for various creative applications in the future. |
| title | DreamDissector: Learning Disentangled Text-to-3D Generation from 2D Diffusion Priors |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.16260 |