SUPER: Selfie Undistortion and Head Pose Editing with Identity Preservation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929390946877440 |
|---|---|
| author | Karpikova, Polina Spiridonov, Andrei Vorontsova, Anna Yaschenko, Anastasia Radionova, Ekaterina Medvedev, Igor Limonov, Alexander |
| author_facet | Karpikova, Polina Spiridonov, Andrei Vorontsova, Anna Yaschenko, Anastasia Radionova, Ekaterina Medvedev, Igor Limonov, Alexander |
| contents | Self-portraits captured from a short distance might look unnatural or even unattractive due to heavy distortions making facial features malformed, and ill-placed head poses. In this paper, we propose SUPER, a novel method of eliminating distortions and adjusting head pose in a close-up face crop. We perform 3D GAN inversion for a facial image by optimizing camera parameters and face latent code, which gives a generated image. Besides, we estimate depth from the obtained latent code, create a depth-induced 3D mesh, and render it with updated camera parameters to obtain a warped portrait. Finally, we apply the visibility-based blending so that visible regions are reprojected, and occluded parts are restored with a generative model. Experiments on face undistortion benchmarks and on our self-collected Head Rotation dataset (HeRo), show that SUPER outperforms previous approaches both qualitatively and quantitatively, opening new possibilities for photorealistic selfie editing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_12700 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SUPER: Selfie Undistortion and Head Pose Editing with Identity Preservation Karpikova, Polina Spiridonov, Andrei Vorontsova, Anna Yaschenko, Anastasia Radionova, Ekaterina Medvedev, Igor Limonov, Alexander Computer Vision and Pattern Recognition Machine Learning Self-portraits captured from a short distance might look unnatural or even unattractive due to heavy distortions making facial features malformed, and ill-placed head poses. In this paper, we propose SUPER, a novel method of eliminating distortions and adjusting head pose in a close-up face crop. We perform 3D GAN inversion for a facial image by optimizing camera parameters and face latent code, which gives a generated image. Besides, we estimate depth from the obtained latent code, create a depth-induced 3D mesh, and render it with updated camera parameters to obtain a warped portrait. Finally, we apply the visibility-based blending so that visible regions are reprojected, and occluded parts are restored with a generative model. Experiments on face undistortion benchmarks and on our self-collected Head Rotation dataset (HeRo), show that SUPER outperforms previous approaches both qualitatively and quantitatively, opening new possibilities for photorealistic selfie editing. |
| title | SUPER: Selfie Undistortion and Head Pose Editing with Identity Preservation |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2406.12700 |