TPA3D: Triplane Attention for Fast Text-to-3D Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914960421945344 |
|---|---|
| author | Wu, Bin-Shih Chen, Hong-En Huang, Sheng-Yu Wang, Yu-Chiang Frank |
| author_facet | Wu, Bin-Shih Chen, Hong-En Huang, Sheng-Yu Wang, Yu-Chiang Frank |
| contents | Due to the lack of large-scale text-3D correspondence data, recent text-to-3D generation works mainly rely on utilizing 2D diffusion models for synthesizing 3D data. Since diffusion-based methods typically require significant optimization time for both training and inference, the use of GAN-based models would still be desirable for fast 3D generation. In this work, we propose Triplane Attention for text-guided 3D generation (TPA3D), an end-to-end trainable GAN-based deep learning model for fast text-to-3D generation. With only 3D shape data and their rendered 2D images observed during training, our TPA3D is designed to retrieve detailed visual descriptions for synthesizing the corresponding 3D mesh data. This is achieved by the proposed attention mechanisms on the extracted sentence and word-level text features. In our experiments, we show that TPA3D generates high-quality 3D textured shapes aligned with fine-grained descriptions, while impressive computation efficiency can be observed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_02647 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | TPA3D: Triplane Attention for Fast Text-to-3D Generation Wu, Bin-Shih Chen, Hong-En Huang, Sheng-Yu Wang, Yu-Chiang Frank Computer Vision and Pattern Recognition Due to the lack of large-scale text-3D correspondence data, recent text-to-3D generation works mainly rely on utilizing 2D diffusion models for synthesizing 3D data. Since diffusion-based methods typically require significant optimization time for both training and inference, the use of GAN-based models would still be desirable for fast 3D generation. In this work, we propose Triplane Attention for text-guided 3D generation (TPA3D), an end-to-end trainable GAN-based deep learning model for fast text-to-3D generation. With only 3D shape data and their rendered 2D images observed during training, our TPA3D is designed to retrieve detailed visual descriptions for synthesizing the corresponding 3D mesh data. This is achieved by the proposed attention mechanisms on the extracted sentence and word-level text features. In our experiments, we show that TPA3D generates high-quality 3D textured shapes aligned with fine-grained descriptions, while impressive computation efficiency can be observed. |
| title | TPA3D: Triplane Attention for Fast Text-to-3D Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.02647 |