LeGO: Leveraging a Surface Deformation Network for Animatable Stylized Face Generation with One Example
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929286068305920 |
|---|---|
| author | Yoon, Soyeon Yun, Kwan Seo, Kwanggyoon Cha, Sihun Yoo, Jung Eun Noh, Junyong |
| author_facet | Yoon, Soyeon Yun, Kwan Seo, Kwanggyoon Cha, Sihun Yoo, Jung Eun Noh, Junyong |
| contents | Recent advances in 3D face stylization have made significant strides in few to zero-shot settings. However, the degree of stylization achieved by existing methods is often not sufficient for practical applications because they are mostly based on statistical 3D Morphable Models (3DMM) with limited variations. To this end, we propose a method that can produce a highly stylized 3D face model with desired topology. Our methods train a surface deformation network with 3DMM and translate its domain to the target style using a paired exemplar. The network achieves stylization of the 3D face mesh by mimicking the style of the target using a differentiable renderer and directional CLIP losses. Additionally, during the inference process, we utilize a Mesh Agnostic Encoder (MAGE) that takes deformation target, a mesh of diverse topologies as input to the stylization process and encodes its shape into our latent space. The resulting stylized face model can be animated by commonly used 3DMM blend shapes. A set of quantitative and qualitative evaluations demonstrate that our method can produce highly stylized face meshes according to a given style and output them in a desired topology. We also demonstrate example applications of our method including image-based stylized avatar generation, linear interpolation of geometric styles, and facial animation of stylized avatars. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_15227 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LeGO: Leveraging a Surface Deformation Network for Animatable Stylized Face Generation with One Example Yoon, Soyeon Yun, Kwan Seo, Kwanggyoon Cha, Sihun Yoo, Jung Eun Noh, Junyong Computer Vision and Pattern Recognition Graphics 68T45 I.4.9 Recent advances in 3D face stylization have made significant strides in few to zero-shot settings. However, the degree of stylization achieved by existing methods is often not sufficient for practical applications because they are mostly based on statistical 3D Morphable Models (3DMM) with limited variations. To this end, we propose a method that can produce a highly stylized 3D face model with desired topology. Our methods train a surface deformation network with 3DMM and translate its domain to the target style using a paired exemplar. The network achieves stylization of the 3D face mesh by mimicking the style of the target using a differentiable renderer and directional CLIP losses. Additionally, during the inference process, we utilize a Mesh Agnostic Encoder (MAGE) that takes deformation target, a mesh of diverse topologies as input to the stylization process and encodes its shape into our latent space. The resulting stylized face model can be animated by commonly used 3DMM blend shapes. A set of quantitative and qualitative evaluations demonstrate that our method can produce highly stylized face meshes according to a given style and output them in a desired topology. We also demonstrate example applications of our method including image-based stylized avatar generation, linear interpolation of geometric styles, and facial animation of stylized avatars. |
| title | LeGO: Leveraging a Surface Deformation Network for Animatable Stylized Face Generation with One Example |
| topic | Computer Vision and Pattern Recognition Graphics 68T45 I.4.9 |
| url | https://arxiv.org/abs/2403.15227 |