Real-time 3D-aware Portrait Editing from a Single Image
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909261161824256 |
|---|---|
| author | Bai, Qingyan Shi, Zifan Xu, Yinghao Ouyang, Hao Wang, Qiuyu Yang, Ceyuan Wang, Xuan Wetzstein, Gordon Shen, Yujun Chen, Qifeng |
| author_facet | Bai, Qingyan Shi, Zifan Xu, Yinghao Ouyang, Hao Wang, Qiuyu Yang, Ceyuan Wang, Xuan Wetzstein, Gordon Shen, Yujun Chen, Qifeng |
| contents | This work presents 3DPE, a practical method that can efficiently edit a face image following given prompts, like reference images or text descriptions, in a 3D-aware manner. To this end, a lightweight module is distilled from a 3D portrait generator and a text-to-image model, which provide prior knowledge of face geometry and superior editing capability, respectively. Such a design brings two compelling advantages over existing approaches. First, our method achieves real-time editing with a feedforward network (i.e., ~0.04s per image), over 100x faster than the second competitor. Second, thanks to the powerful priors, our module could focus on the learning of editing-related variations, such that it manages to handle various types of editing simultaneously in the training phase and further supports fast adaptation to user-specified customized types of editing during inference (e.g., with ~5min fine-tuning per style). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_14000 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Real-time 3D-aware Portrait Editing from a Single Image Bai, Qingyan Shi, Zifan Xu, Yinghao Ouyang, Hao Wang, Qiuyu Yang, Ceyuan Wang, Xuan Wetzstein, Gordon Shen, Yujun Chen, Qifeng Computer Vision and Pattern Recognition This work presents 3DPE, a practical method that can efficiently edit a face image following given prompts, like reference images or text descriptions, in a 3D-aware manner. To this end, a lightweight module is distilled from a 3D portrait generator and a text-to-image model, which provide prior knowledge of face geometry and superior editing capability, respectively. Such a design brings two compelling advantages over existing approaches. First, our method achieves real-time editing with a feedforward network (i.e., ~0.04s per image), over 100x faster than the second competitor. Second, thanks to the powerful priors, our module could focus on the learning of editing-related variations, such that it manages to handle various types of editing simultaneously in the training phase and further supports fast adaptation to user-specified customized types of editing during inference (e.g., with ~5min fine-tuning per style). |
| title | Real-time 3D-aware Portrait Editing from a Single Image |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.14000 |