S3Editor: A Sparse Semantic-Disentangled Self-Training Framework for Face Video Editing
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914750363860992 |
|---|---|
| author | Wang, Guangzhi Chen, Tianyi Ghasedi, Kamran Wu, HsiangTao Ding, Tianyu Nuesmeyer, Chris Zharkov, Ilya Kankanhalli, Mohan Liang, Luming |
| author_facet | Wang, Guangzhi Chen, Tianyi Ghasedi, Kamran Wu, HsiangTao Ding, Tianyu Nuesmeyer, Chris Zharkov, Ilya Kankanhalli, Mohan Liang, Luming |
| contents | Face attribute editing plays a pivotal role in various applications. However, existing methods encounter challenges in achieving high-quality results while preserving identity, editing faithfulness, and temporal consistency. These challenges are rooted in issues related to the training pipeline, including limited supervision, architecture design, and optimization strategy. In this work, we introduce S3Editor, a Sparse Semantic-disentangled Self-training framework for face video editing. S3Editor is a generic solution that comprehensively addresses these challenges with three key contributions. Firstly, S3Editor adopts a self-training paradigm to enhance the training process through semi-supervision. Secondly, we propose a semantic disentangled architecture with a dynamic routing mechanism that accommodates diverse editing requirements. Thirdly, we present a structured sparse optimization schema that identifies and deactivates malicious neurons to further disentangle impacts from untarget attributes. S3Editor is model-agnostic and compatible with various editing approaches. Our extensive qualitative and quantitative results affirm that our approach significantly enhances identity preservation, editing fidelity, as well as temporal consistency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_08111 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | S3Editor: A Sparse Semantic-Disentangled Self-Training Framework for Face Video Editing Wang, Guangzhi Chen, Tianyi Ghasedi, Kamran Wu, HsiangTao Ding, Tianyu Nuesmeyer, Chris Zharkov, Ilya Kankanhalli, Mohan Liang, Luming Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Face attribute editing plays a pivotal role in various applications. However, existing methods encounter challenges in achieving high-quality results while preserving identity, editing faithfulness, and temporal consistency. These challenges are rooted in issues related to the training pipeline, including limited supervision, architecture design, and optimization strategy. In this work, we introduce S3Editor, a Sparse Semantic-disentangled Self-training framework for face video editing. S3Editor is a generic solution that comprehensively addresses these challenges with three key contributions. Firstly, S3Editor adopts a self-training paradigm to enhance the training process through semi-supervision. Secondly, we propose a semantic disentangled architecture with a dynamic routing mechanism that accommodates diverse editing requirements. Thirdly, we present a structured sparse optimization schema that identifies and deactivates malicious neurons to further disentangle impacts from untarget attributes. S3Editor is model-agnostic and compatible with various editing approaches. Our extensive qualitative and quantitative results affirm that our approach significantly enhances identity preservation, editing fidelity, as well as temporal consistency. |
| title | S3Editor: A Sparse Semantic-Disentangled Self-Training Framework for Face Video Editing |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2404.08111 |