Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.05577 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909246000463872 |
|---|---|
| author | Su, Jiacheng Liu, Kunhong Chen, Liyan Yao, Junfeng Liu, Qingsong Lv, Dongdong |
| author_facet | Su, Jiacheng Liu, Kunhong Chen, Liyan Yao, Junfeng Liu, Qingsong Lv, Dongdong |
| contents | The existing methods for audio-driven talking head video editing have the limitations of poor visual effects. This paper tries to tackle this problem through editing talking face images seamless with different emotions based on two modules: (1) an audio-to-landmark module, consisting of the CrossReconstructed Emotion Disentanglement and an alignment network module. It bridges the gap between speech and facial motions by predicting corresponding emotional landmarks from speech; (2) a landmark-based editing module edits face videos via StyleGAN. It aims to generate the seamless edited video consisting of the emotion and content components from the input audio. Extensive experiments confirm that compared with state-of-the-arts methods, our method provides high-resolution videos with high visual quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_05577 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Audio-driven High-resolution Seamless Talking Head Video Editing via StyleGAN Su, Jiacheng Liu, Kunhong Chen, Liyan Yao, Junfeng Liu, Qingsong Lv, Dongdong Computer Vision and Pattern Recognition The existing methods for audio-driven talking head video editing have the limitations of poor visual effects. This paper tries to tackle this problem through editing talking face images seamless with different emotions based on two modules: (1) an audio-to-landmark module, consisting of the CrossReconstructed Emotion Disentanglement and an alignment network module. It bridges the gap between speech and facial motions by predicting corresponding emotional landmarks from speech; (2) a landmark-based editing module edits face videos via StyleGAN. It aims to generate the seamless edited video consisting of the emotion and content components from the input audio. Extensive experiments confirm that compared with state-of-the-arts methods, our method provides high-resolution videos with high visual quality. |
| title | Audio-driven High-resolution Seamless Talking Head Video Editing via StyleGAN |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.05577 |