GenesisTex: Adapting Image Denoising Diffusion to Texture Space
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917622727049216 |
|---|---|
| author | Gao, Chenjian Jiang, Boyan Li, Xinghui Zhang, Yingpeng Yu, Qian |
| author_facet | Gao, Chenjian Jiang, Boyan Li, Xinghui Zhang, Yingpeng Yu, Qian |
| contents | We present GenesisTex, a novel method for synthesizing textures for 3D geometries from text descriptions. GenesisTex adapts the pretrained image diffusion model to texture space by texture space sampling. Specifically, we maintain a latent texture map for each viewpoint, which is updated with predicted noise on the rendering of the corresponding viewpoint. The sampled latent texture maps are then decoded into a final texture map. During the sampling process, we focus on both global and local consistency across multiple viewpoints: global consistency is achieved through the integration of style consistency mechanisms within the noise prediction network, and low-level consistency is achieved by dynamically aligning latent textures. Finally, we apply reference-based inpainting and img2img on denser views for texture refinement. Our approach overcomes the limitations of slow optimization in distillation-based methods and instability in inpainting-based methods. Experiments on meshes from various sources demonstrate that our method surpasses the baseline methods quantitatively and qualitatively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_17782 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GenesisTex: Adapting Image Denoising Diffusion to Texture Space Gao, Chenjian Jiang, Boyan Li, Xinghui Zhang, Yingpeng Yu, Qian Computer Vision and Pattern Recognition Graphics We present GenesisTex, a novel method for synthesizing textures for 3D geometries from text descriptions. GenesisTex adapts the pretrained image diffusion model to texture space by texture space sampling. Specifically, we maintain a latent texture map for each viewpoint, which is updated with predicted noise on the rendering of the corresponding viewpoint. The sampled latent texture maps are then decoded into a final texture map. During the sampling process, we focus on both global and local consistency across multiple viewpoints: global consistency is achieved through the integration of style consistency mechanisms within the noise prediction network, and low-level consistency is achieved by dynamically aligning latent textures. Finally, we apply reference-based inpainting and img2img on denser views for texture refinement. Our approach overcomes the limitations of slow optimization in distillation-based methods and instability in inpainting-based methods. Experiments on meshes from various sources demonstrate that our method surpasses the baseline methods quantitatively and qualitatively. |
| title | GenesisTex: Adapting Image Denoising Diffusion to Texture Space |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2403.17782 |