TEXGen: a Generative Diffusion Model for Mesh Textures
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917844847951872 |
|---|---|
| author | Yu, Xin Yuan, Ze Guo, Yuan-Chen Liu, Ying-Tian Liu, JianHui Li, Yangguang Cao, Yan-Pei Liang, Ding Qi, Xiaojuan |
| author_facet | Yu, Xin Yuan, Ze Guo, Yuan-Chen Liu, Ying-Tian Liu, JianHui Li, Yangguang Cao, Yan-Pei Liang, Ding Qi, Xiaojuan |
| contents | While high-quality texture maps are essential for realistic 3D asset rendering, few studies have explored learning directly in the texture space, especially on large-scale datasets. In this work, we depart from the conventional approach of relying on pre-trained 2D diffusion models for test-time optimization of 3D textures. Instead, we focus on the fundamental problem of learning in the UV texture space itself. For the first time, we train a large diffusion model capable of directly generating high-resolution texture maps in a feed-forward manner. To facilitate efficient learning in high-resolution UV spaces, we propose a scalable network architecture that interleaves convolutions on UV maps with attention layers on point clouds. Leveraging this architectural design, we train a 700 million parameter diffusion model that can generate UV texture maps guided by text prompts and single-view images. Once trained, our model naturally supports various extended applications, including text-guided texture inpainting, sparse-view texture completion, and text-driven texture synthesis. Project page is at http://cvmi-lab.github.io/TEXGen/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_14740 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TEXGen: a Generative Diffusion Model for Mesh Textures Yu, Xin Yuan, Ze Guo, Yuan-Chen Liu, Ying-Tian Liu, JianHui Li, Yangguang Cao, Yan-Pei Liang, Ding Qi, Xiaojuan Computer Vision and Pattern Recognition Artificial Intelligence Graphics While high-quality texture maps are essential for realistic 3D asset rendering, few studies have explored learning directly in the texture space, especially on large-scale datasets. In this work, we depart from the conventional approach of relying on pre-trained 2D diffusion models for test-time optimization of 3D textures. Instead, we focus on the fundamental problem of learning in the UV texture space itself. For the first time, we train a large diffusion model capable of directly generating high-resolution texture maps in a feed-forward manner. To facilitate efficient learning in high-resolution UV spaces, we propose a scalable network architecture that interleaves convolutions on UV maps with attention layers on point clouds. Leveraging this architectural design, we train a 700 million parameter diffusion model that can generate UV texture maps guided by text prompts and single-view images. Once trained, our model naturally supports various extended applications, including text-guided texture inpainting, sparse-view texture completion, and text-driven texture synthesis. Project page is at http://cvmi-lab.github.io/TEXGen/. |
| title | TEXGen: a Generative Diffusion Model for Mesh Textures |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Graphics |
| url | https://arxiv.org/abs/2411.14740 |