AToM: Amortized Text-to-Mesh using 2D Diffusion
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929231148089344 |
|---|---|
| author | Qian, Guocheng Cao, Junli Siarohin, Aliaksandr Kant, Yash Wang, Chaoyang Vasilkovsky, Michael Lee, Hsin-Ying Fang, Yuwei Skorokhodov, Ivan Zhuang, Peiye Gilitschenski, Igor Ren, Jian Ghanem, Bernard Aberman, Kfir Tulyakov, Sergey |
| author_facet | Qian, Guocheng Cao, Junli Siarohin, Aliaksandr Kant, Yash Wang, Chaoyang Vasilkovsky, Michael Lee, Hsin-Ying Fang, Yuwei Skorokhodov, Ivan Zhuang, Peiye Gilitschenski, Igor Ren, Jian Ghanem, Bernard Aberman, Kfir Tulyakov, Sergey |
| contents | We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh framework optimized across multiple text prompts simultaneously. In contrast to existing text-to-3D methods that often entail time-consuming per-prompt optimization and commonly output representations other than polygonal meshes, AToM directly generates high-quality textured meshes in less than 1 second with around 10 times reduction in the training cost, and generalizes to unseen prompts. Our key idea is a novel triplane-based text-to-mesh architecture with a two-stage amortized optimization strategy that ensures stable training and enables scalability. Through extensive experiments on various prompt benchmarks, AToM significantly outperforms state-of-the-art amortized approaches with over 4 times higher accuracy (in DF415 dataset) and produces more distinguishable and higher-quality 3D outputs. AToM demonstrates strong generalizability, offering finegrained 3D assets for unseen interpolated prompts without further optimization during inference, unlike per-prompt solutions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_00867 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AToM: Amortized Text-to-Mesh using 2D Diffusion Qian, Guocheng Cao, Junli Siarohin, Aliaksandr Kant, Yash Wang, Chaoyang Vasilkovsky, Michael Lee, Hsin-Ying Fang, Yuwei Skorokhodov, Ivan Zhuang, Peiye Gilitschenski, Igor Ren, Jian Ghanem, Bernard Aberman, Kfir Tulyakov, Sergey Computer Vision and Pattern Recognition We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh framework optimized across multiple text prompts simultaneously. In contrast to existing text-to-3D methods that often entail time-consuming per-prompt optimization and commonly output representations other than polygonal meshes, AToM directly generates high-quality textured meshes in less than 1 second with around 10 times reduction in the training cost, and generalizes to unseen prompts. Our key idea is a novel triplane-based text-to-mesh architecture with a two-stage amortized optimization strategy that ensures stable training and enables scalability. Through extensive experiments on various prompt benchmarks, AToM significantly outperforms state-of-the-art amortized approaches with over 4 times higher accuracy (in DF415 dataset) and produces more distinguishable and higher-quality 3D outputs. AToM demonstrates strong generalizability, offering finegrained 3D assets for unseen interpolated prompts without further optimization during inference, unlike per-prompt solutions. |
| title | AToM: Amortized Text-to-Mesh using 2D Diffusion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.00867 |