Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.02293 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912200508047360 |
|---|---|
| author | Yang, Xianghui Shi, Huiwen Zhang, Bowen Yang, Fan Wang, Jiacheng Zhao, Hongxu Liu, Xinhai Wang, Xinzhou Lin, Qingxiang Yu, Jiaao Wang, Lifu Xu, Jing He, Zebin Chen, Zhuo Liu, Sicong Wu, Junta Lian, Yihang Yang, Shaoxiong Liu, Yuhong Yang, Yong Wang, Di Jiang, Jie Guo, Chunchao |
| author_facet | Yang, Xianghui Shi, Huiwen Zhang, Bowen Yang, Fan Wang, Jiacheng Zhao, Hongxu Liu, Xinhai Wang, Xinzhou Lin, Qingxiang Yu, Jiaao Wang, Lifu Xu, Jing He, Zebin Chen, Zhuo Liu, Sicong Wu, Junta Lian, Yihang Yang, Shaoxiong Liu, Yuhong Yang, Yong Wang, Di Jiang, Jie Guo, Chunchao |
| contents | While 3D generative models have greatly improved artists' workflows, the existing diffusion models for 3D generation suffer from slow generation and poor generalization. To address this issue, we propose a two-stage approach named Hunyuan3D 1.0 including a lite version and a standard version, that both support text- and image-conditioned generation. In the first stage, we employ a multi-view diffusion model that efficiently generates multi-view RGB in approximately 4 seconds. These multi-view images capture rich details of the 3D asset from different viewpoints, relaxing the tasks from single-view to multi-view reconstruction. In the second stage, we introduce a feed-forward reconstruction model that rapidly and faithfully reconstructs the 3D asset given the generated multi-view images in approximately 7 seconds. The reconstruction network learns to handle noises and in-consistency introduced by the multi-view diffusion and leverages the available information from the condition image to efficiently recover the 3D structure. Our framework involves the text-to-image model, i.e., Hunyuan-DiT, making it a unified framework to support both text- and image-conditioned 3D generation. Our standard version has 3x more parameters than our lite and other existing model. Our Hunyuan3D 1.0 achieves an impressive balance between speed and quality, significantly reducing generation time while maintaining the quality and diversity of the produced assets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_02293 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation Yang, Xianghui Shi, Huiwen Zhang, Bowen Yang, Fan Wang, Jiacheng Zhao, Hongxu Liu, Xinhai Wang, Xinzhou Lin, Qingxiang Yu, Jiaao Wang, Lifu Xu, Jing He, Zebin Chen, Zhuo Liu, Sicong Wu, Junta Lian, Yihang Yang, Shaoxiong Liu, Yuhong Yang, Yong Wang, Di Jiang, Jie Guo, Chunchao Computer Vision and Pattern Recognition Artificial Intelligence While 3D generative models have greatly improved artists' workflows, the existing diffusion models for 3D generation suffer from slow generation and poor generalization. To address this issue, we propose a two-stage approach named Hunyuan3D 1.0 including a lite version and a standard version, that both support text- and image-conditioned generation. In the first stage, we employ a multi-view diffusion model that efficiently generates multi-view RGB in approximately 4 seconds. These multi-view images capture rich details of the 3D asset from different viewpoints, relaxing the tasks from single-view to multi-view reconstruction. In the second stage, we introduce a feed-forward reconstruction model that rapidly and faithfully reconstructs the 3D asset given the generated multi-view images in approximately 7 seconds. The reconstruction network learns to handle noises and in-consistency introduced by the multi-view diffusion and leverages the available information from the condition image to efficiently recover the 3D structure. Our framework involves the text-to-image model, i.e., Hunyuan-DiT, making it a unified framework to support both text- and image-conditioned 3D generation. Our standard version has 3x more parameters than our lite and other existing model. Our Hunyuan3D 1.0 achieves an impressive balance between speed and quality, significantly reducing generation time while maintaining the quality and diversity of the produced assets. |
| title | Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2411.02293 |