3DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909192027111424 |
|---|---|
| author | Hong, Fangzhou Tang, Jiaxiang Cao, Ziang Shi, Min Wu, Tong Chen, Zhaoxi Yang, Shuai Wang, Tengfei Pan, Liang Lin, Dahua Liu, Ziwei |
| author_facet | Hong, Fangzhou Tang, Jiaxiang Cao, Ziang Shi, Min Wu, Tong Chen, Zhaoxi Yang, Shuai Wang, Tengfei Pan, Liang Lin, Dahua Liu, Ziwei |
| contents | We present a two-stage text-to-3D generation system, namely 3DTopia, which generates high-quality general 3D assets within 5 minutes using hybrid diffusion priors. The first stage samples from a 3D diffusion prior directly learned from 3D data. Specifically, it is powered by a text-conditioned tri-plane latent diffusion model, which quickly generates coarse 3D samples for fast prototyping. The second stage utilizes 2D diffusion priors to further refine the texture of coarse 3D models from the first stage. The refinement consists of both latent and pixel space optimization for high-quality texture generation. To facilitate the training of the proposed system, we clean and caption the largest open-source 3D dataset, Objaverse, by combining the power of vision language models and large language models. Experiment results are reported qualitatively and quantitatively to show the performance of the proposed system. Our codes and models are available at https://github.com/3DTopia/3DTopia |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_02234 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | 3DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors Hong, Fangzhou Tang, Jiaxiang Cao, Ziang Shi, Min Wu, Tong Chen, Zhaoxi Yang, Shuai Wang, Tengfei Pan, Liang Lin, Dahua Liu, Ziwei Computer Vision and Pattern Recognition We present a two-stage text-to-3D generation system, namely 3DTopia, which generates high-quality general 3D assets within 5 minutes using hybrid diffusion priors. The first stage samples from a 3D diffusion prior directly learned from 3D data. Specifically, it is powered by a text-conditioned tri-plane latent diffusion model, which quickly generates coarse 3D samples for fast prototyping. The second stage utilizes 2D diffusion priors to further refine the texture of coarse 3D models from the first stage. The refinement consists of both latent and pixel space optimization for high-quality texture generation. To facilitate the training of the proposed system, we clean and caption the largest open-source 3D dataset, Objaverse, by combining the power of vision language models and large language models. Experiment results are reported qualitatively and quantitatively to show the performance of the proposed system. Our codes and models are available at https://github.com/3DTopia/3DTopia |
| title | 3DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.02234 |