GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913295900868608 |
|---|---|
| author | Bao, Chong Zhang, Yinda Li, Yuan Zhang, Xiyu Yang, Bangbang Bao, Hujun Pollefeys, Marc Zhang, Guofeng Cui, Zhaopeng |
| author_facet | Bao, Chong Zhang, Yinda Li, Yuan Zhang, Xiyu Yang, Bangbang Bao, Hujun Pollefeys, Marc Zhang, Guofeng Cui, Zhaopeng |
| contents | Recently, we have witnessed the explosive growth of various volumetric representations in modeling animatable head avatars. However, due to the diversity of frameworks, there is no practical method to support high-level applications like 3D head avatar editing across different representations. In this paper, we propose a generic avatar editing approach that can be universally applied to various 3DMM driving volumetric head avatars. To achieve this goal, we design a novel expression-aware modification generative model, which enables lift 2D editing from a single image to a consistent 3D modification field. To ensure the effectiveness of the generative modification process, we develop several techniques, including an expression-dependent modification distillation scheme to draw knowledge from the large-scale head avatar model and 2D facial texture editing tools, implicit latent space guidance to enhance model convergence, and a segmentation-based loss reweight strategy for fine-grained texture inversion. Extensive experiments demonstrate that our method delivers high-quality and consistent results across multiple expression and viewpoints. Project page: https://zju3dv.github.io/geneavatar/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_02152 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image Bao, Chong Zhang, Yinda Li, Yuan Zhang, Xiyu Yang, Bangbang Bao, Hujun Pollefeys, Marc Zhang, Guofeng Cui, Zhaopeng Computer Vision and Pattern Recognition Recently, we have witnessed the explosive growth of various volumetric representations in modeling animatable head avatars. However, due to the diversity of frameworks, there is no practical method to support high-level applications like 3D head avatar editing across different representations. In this paper, we propose a generic avatar editing approach that can be universally applied to various 3DMM driving volumetric head avatars. To achieve this goal, we design a novel expression-aware modification generative model, which enables lift 2D editing from a single image to a consistent 3D modification field. To ensure the effectiveness of the generative modification process, we develop several techniques, including an expression-dependent modification distillation scheme to draw knowledge from the large-scale head avatar model and 2D facial texture editing tools, implicit latent space guidance to enhance model convergence, and a segmentation-based loss reweight strategy for fine-grained texture inversion. Extensive experiments demonstrate that our method delivers high-quality and consistent results across multiple expression and viewpoints. Project page: https://zju3dv.github.io/geneavatar/ |
| title | GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.02152 |