Visual Style Prompt Learning Using Diffusion Models for Blind Face Restoration
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912172840321024 |
|---|---|
| author | Lu, Wanglong Wang, Jikai Wang, Tao Zhang, Kaihao Jiang, Xianta Zhao, Hanli |
| author_facet | Lu, Wanglong Wang, Jikai Wang, Tao Zhang, Kaihao Jiang, Xianta Zhao, Hanli |
| contents | Blind face restoration aims to recover high-quality facial images from various unidentified sources of degradation, posing significant challenges due to the minimal information retrievable from the degraded images. Prior knowledge-based methods, leveraging geometric priors and facial features, have led to advancements in face restoration but often fall short of capturing fine details. To address this, we introduce a visual style prompt learning framework that utilizes diffusion probabilistic models to explicitly generate visual prompts within the latent space of pre-trained generative models. These prompts are designed to guide the restoration process. To fully utilize the visual prompts and enhance the extraction of informative and rich patterns, we introduce a style-modulated aggregation transformation layer. Extensive experiments and applications demonstrate the superiority of our method in achieving high-quality blind face restoration. The source code is available at \href{https://github.com/LonglongaaaGo/VSPBFR}{https://github.com/LonglongaaaGo/VSPBFR}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_21042 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Visual Style Prompt Learning Using Diffusion Models for Blind Face Restoration Lu, Wanglong Wang, Jikai Wang, Tao Zhang, Kaihao Jiang, Xianta Zhao, Hanli Computer Vision and Pattern Recognition Multimedia 68U10 I.4.3; I.4.4; I.4.5; I.4.9 Blind face restoration aims to recover high-quality facial images from various unidentified sources of degradation, posing significant challenges due to the minimal information retrievable from the degraded images. Prior knowledge-based methods, leveraging geometric priors and facial features, have led to advancements in face restoration but often fall short of capturing fine details. To address this, we introduce a visual style prompt learning framework that utilizes diffusion probabilistic models to explicitly generate visual prompts within the latent space of pre-trained generative models. These prompts are designed to guide the restoration process. To fully utilize the visual prompts and enhance the extraction of informative and rich patterns, we introduce a style-modulated aggregation transformation layer. Extensive experiments and applications demonstrate the superiority of our method in achieving high-quality blind face restoration. The source code is available at \href{https://github.com/LonglongaaaGo/VSPBFR}{https://github.com/LonglongaaaGo/VSPBFR}. |
| title | Visual Style Prompt Learning Using Diffusion Models for Blind Face Restoration |
| topic | Computer Vision and Pattern Recognition Multimedia 68U10 I.4.3; I.4.4; I.4.5; I.4.9 |
| url | https://arxiv.org/abs/2412.21042 |