PersonaMagic: Stage-Regulated High-Fidelity Face Customization with Tandem Equilibrium
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913621054849024 |
|---|---|
| author | Li, Xinzhe Zhan, Jiahui He, Shengfeng Xu, Yangyang Dong, Junyu Zhang, Huaidong Du, Yong |
| author_facet | Li, Xinzhe Zhan, Jiahui He, Shengfeng Xu, Yangyang Dong, Junyu Zhang, Huaidong Du, Yong |
| contents | Personalized image generation has made significant strides in adapting content to novel concepts. However, a persistent challenge remains: balancing the accurate reconstruction of unseen concepts with the need for editability according to the prompt, especially when dealing with the complex nuances of facial features. In this study, we delve into the temporal dynamics of the text-to-image conditioning process, emphasizing the crucial role of stage partitioning in introducing new concepts. We present PersonaMagic, a stage-regulated generative technique designed for high-fidelity face customization. Using a simple MLP network, our method learns a series of embeddings within a specific timestep interval to capture face concepts. Additionally, we develop a Tandem Equilibrium mechanism that adjusts self-attention responses in the text encoder, balancing text description and identity preservation, improving both areas. Extensive experiments confirm the superiority of PersonaMagic over state-of-the-art methods in both qualitative and quantitative evaluations. Moreover, its robustness and flexibility are validated in non-facial domains, and it can also serve as a valuable plug-in for enhancing the performance of pretrained personalization models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_15674 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PersonaMagic: Stage-Regulated High-Fidelity Face Customization with Tandem Equilibrium Li, Xinzhe Zhan, Jiahui He, Shengfeng Xu, Yangyang Dong, Junyu Zhang, Huaidong Du, Yong Computer Vision and Pattern Recognition Personalized image generation has made significant strides in adapting content to novel concepts. However, a persistent challenge remains: balancing the accurate reconstruction of unseen concepts with the need for editability according to the prompt, especially when dealing with the complex nuances of facial features. In this study, we delve into the temporal dynamics of the text-to-image conditioning process, emphasizing the crucial role of stage partitioning in introducing new concepts. We present PersonaMagic, a stage-regulated generative technique designed for high-fidelity face customization. Using a simple MLP network, our method learns a series of embeddings within a specific timestep interval to capture face concepts. Additionally, we develop a Tandem Equilibrium mechanism that adjusts self-attention responses in the text encoder, balancing text description and identity preservation, improving both areas. Extensive experiments confirm the superiority of PersonaMagic over state-of-the-art methods in both qualitative and quantitative evaluations. Moreover, its robustness and flexibility are validated in non-facial domains, and it can also serve as a valuable plug-in for enhancing the performance of pretrained personalization models. |
| title | PersonaMagic: Stage-Regulated High-Fidelity Face Customization with Tandem Equilibrium |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.15674 |