OSDFace: One-Step Diffusion Model for Face Restoration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915932539977728 |
|---|---|
| author | Wang, Jingkai Gong, Jue Zhang, Lin Chen, Zheng Liu, Xing Gu, Hong Liu, Yutong Zhang, Yulun Yang, Xiaokang |
| author_facet | Wang, Jingkai Gong, Jue Zhang, Lin Chen, Zheng Liu, Xing Gu, Hong Liu, Yutong Zhang, Yulun Yang, Xiaokang |
| contents | Diffusion models have demonstrated impressive performance in face restoration. Yet, their multi-step inference process remains computationally intensive, limiting their applicability in real-world scenarios. Moreover, existing methods often struggle to generate face images that are harmonious, realistic, and consistent with the subject's identity. In this work, we propose OSDFace, a novel one-step diffusion model for face restoration. Specifically, we propose a visual representation embedder (VRE) to better capture prior information and understand the input face. In VRE, low-quality faces are processed by a visual tokenizer and subsequently embedded with a vector-quantized dictionary to generate visual prompts. Additionally, we incorporate a facial identity loss derived from face recognition to further ensure identity consistency. We further employ a generative adversarial network (GAN) as a guidance model to encourage distribution alignment between the restored face and the ground truth. Experimental results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics, generating high-fidelity, natural face images with high identity consistency. The code and model will be released at https://github.com/jkwang28/OSDFace. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_17163 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OSDFace: One-Step Diffusion Model for Face Restoration Wang, Jingkai Gong, Jue Zhang, Lin Chen, Zheng Liu, Xing Gu, Hong Liu, Yutong Zhang, Yulun Yang, Xiaokang Computer Vision and Pattern Recognition Diffusion models have demonstrated impressive performance in face restoration. Yet, their multi-step inference process remains computationally intensive, limiting their applicability in real-world scenarios. Moreover, existing methods often struggle to generate face images that are harmonious, realistic, and consistent with the subject's identity. In this work, we propose OSDFace, a novel one-step diffusion model for face restoration. Specifically, we propose a visual representation embedder (VRE) to better capture prior information and understand the input face. In VRE, low-quality faces are processed by a visual tokenizer and subsequently embedded with a vector-quantized dictionary to generate visual prompts. Additionally, we incorporate a facial identity loss derived from face recognition to further ensure identity consistency. We further employ a generative adversarial network (GAN) as a guidance model to encourage distribution alignment between the restored face and the ground truth. Experimental results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics, generating high-fidelity, natural face images with high identity consistency. The code and model will be released at https://github.com/jkwang28/OSDFace. |
| title | OSDFace: One-Step Diffusion Model for Face Restoration |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.17163 |