HeadEvolver: Text to Head Avatars via Expressive and Attribute-Preserving Mesh Deformation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913812582498304 |
|---|---|
| author | Wang, Duotun Meng, Hengyu Cai, Zeyu Shao, Zhijing Liu, Qianxi Wang, Lin Fan, Mingming Zhan, Xiaohang Wang, Zeyu |
| author_facet | Wang, Duotun Meng, Hengyu Cai, Zeyu Shao, Zhijing Liu, Qianxi Wang, Lin Fan, Mingming Zhan, Xiaohang Wang, Zeyu |
| contents | Current text-to-avatar methods often rely on implicit representations (e.g., NeRF, SDF, and DMTet), leading to 3D content that artists cannot easily edit and animate in graphics software. This paper introduces a novel framework for generating stylized head avatars from text guidance, which leverages locally learnable mesh deformation and 2D diffusion priors to achieve high-quality digital assets for attribute-preserving manipulation. Given a template mesh, our method represents mesh deformation with per-face Jacobians and adaptively modulates local deformation using a learnable vector field. This vector field enables anisotropic scaling while preserving the rotation of vertices, which can better express identity and geometric details. We employ landmark- and contour-based regularization terms to balance the expressiveness and plausibility of generated avatars from multiple views without relying on any specific shape prior. Our framework can generate realistic shapes and textures that can be further edited via text, while supporting seamless editing using the preserved attributes from the template mesh, such as 3DMM parameters, blendshapes, and UV coordinates. Extensive experiments demonstrate that our framework can generate diverse and expressive head avatars with high-quality meshes that artists can easily manipulate in graphics software, facilitating downstream applications such as efficient asset creation and animation with preserved attributes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_09326 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HeadEvolver: Text to Head Avatars via Expressive and Attribute-Preserving Mesh Deformation Wang, Duotun Meng, Hengyu Cai, Zeyu Shao, Zhijing Liu, Qianxi Wang, Lin Fan, Mingming Zhan, Xiaohang Wang, Zeyu Graphics Artificial Intelligence I.2.6; I.3.8 Current text-to-avatar methods often rely on implicit representations (e.g., NeRF, SDF, and DMTet), leading to 3D content that artists cannot easily edit and animate in graphics software. This paper introduces a novel framework for generating stylized head avatars from text guidance, which leverages locally learnable mesh deformation and 2D diffusion priors to achieve high-quality digital assets for attribute-preserving manipulation. Given a template mesh, our method represents mesh deformation with per-face Jacobians and adaptively modulates local deformation using a learnable vector field. This vector field enables anisotropic scaling while preserving the rotation of vertices, which can better express identity and geometric details. We employ landmark- and contour-based regularization terms to balance the expressiveness and plausibility of generated avatars from multiple views without relying on any specific shape prior. Our framework can generate realistic shapes and textures that can be further edited via text, while supporting seamless editing using the preserved attributes from the template mesh, such as 3DMM parameters, blendshapes, and UV coordinates. Extensive experiments demonstrate that our framework can generate diverse and expressive head avatars with high-quality meshes that artists can easily manipulate in graphics software, facilitating downstream applications such as efficient asset creation and animation with preserved attributes. |
| title | HeadEvolver: Text to Head Avatars via Expressive and Attribute-Preserving Mesh Deformation |
| topic | Graphics Artificial Intelligence I.2.6; I.3.8 |
| url | https://arxiv.org/abs/2403.09326 |