Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.10098 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910368394117120 |
|---|---|
| author | Gao, Nan Li, Jia Huang, Huaibo Zeng, Zhi Shang, Ke Zhang, Shuwu He, Ran |
| author_facet | Gao, Nan Li, Jia Huang, Huaibo Zeng, Zhi Shang, Ke Zhang, Shuwu He, Ran |
| contents | Blind face restoration (BFR) is a highly challenging problem due to the uncertainty of degradation patterns. Current methods have low generalization across photorealistic and heterogeneous domains. In this paper, we propose a Diffusion-Information-Diffusion (DID) framework to tackle diffusion manifold hallucination correction (DiffMAC), which achieves high-generalization face restoration in diverse degraded scenes and heterogeneous domains. Specifically, the first diffusion stage aligns the restored face with spatial feature embedding of the low-quality face based on AdaIN, which synthesizes degradation-removal results but with uncontrollable artifacts for some hard cases. Based on Stage I, Stage II considers information compression using manifold information bottleneck (MIB) and finetunes the first diffusion model to improve facial fidelity. DiffMAC effectively fights against blind degradation patterns and synthesizes high-quality faces with attribute and identity consistencies. Experimental results demonstrate the superiority of DiffMAC over state-of-the-art methods, with a high degree of generalization in real-world and heterogeneous settings. The source code and models will be public. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_10098 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DiffMAC: Diffusion Manifold Hallucination Correction for High Generalization Blind Face Restoration Gao, Nan Li, Jia Huang, Huaibo Zeng, Zhi Shang, Ke Zhang, Shuwu He, Ran Computer Vision and Pattern Recognition Blind face restoration (BFR) is a highly challenging problem due to the uncertainty of degradation patterns. Current methods have low generalization across photorealistic and heterogeneous domains. In this paper, we propose a Diffusion-Information-Diffusion (DID) framework to tackle diffusion manifold hallucination correction (DiffMAC), which achieves high-generalization face restoration in diverse degraded scenes and heterogeneous domains. Specifically, the first diffusion stage aligns the restored face with spatial feature embedding of the low-quality face based on AdaIN, which synthesizes degradation-removal results but with uncontrollable artifacts for some hard cases. Based on Stage I, Stage II considers information compression using manifold information bottleneck (MIB) and finetunes the first diffusion model to improve facial fidelity. DiffMAC effectively fights against blind degradation patterns and synthesizes high-quality faces with attribute and identity consistencies. Experimental results demonstrate the superiority of DiffMAC over state-of-the-art methods, with a high degree of generalization in real-world and heterogeneous settings. The source code and models will be public. |
| title | DiffMAC: Diffusion Manifold Hallucination Correction for High Generalization Blind Face Restoration |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.10098 |