Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.13355 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911007415205888 |
|---|---|
| author | Chen, Yan Shang, Hanlin Liu, Ce Chen, Yuxuan Li, Hui Yuan, Weihao Zhu, Hao Dong, Zilong Zhu, Siyu |
| author_facet | Chen, Yan Shang, Hanlin Liu, Ce Chen, Yuxuan Li, Hui Yuan, Weihao Zhu, Hao Dong, Zilong Zhu, Siyu |
| contents | Video face restoration faces a critical challenge in maintaining temporal consistency while recovering fine facial details from degraded inputs. This paper presents a novel approach that extends Vector-Quantized Variational Autoencoders (VQ-VAEs), pretrained on static high-quality portraits, into a video restoration framework through variational latent space modeling. Our key innovation lies in reformulating discrete codebook representations as Dirichlet-distributed continuous variables, enabling probabilistic transitions between facial features across frames. A spatio-temporal Transformer architecture jointly models inter-frame dependencies and predicts latent distributions, while a Laplacian-constrained reconstruction loss combined with perceptual (LPIPS) regularization enhances both pixel accuracy and visual quality. Comprehensive evaluations on blind face restoration, video inpainting, and facial colorization tasks demonstrate state-of-the-art performance. This work establishes an effective paradigm for adapting intensive image priors, pretrained on high-quality images, to video restoration while addressing the critical challenge of flicker artifacts. The source code has been open-sourced and is available at https://github.com/fudan-generative-vision/DicFace. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_13355 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DicFace: Dirichlet-Constrained Variational Codebook Learning for Temporally Coherent Video Face Restoration Chen, Yan Shang, Hanlin Liu, Ce Chen, Yuxuan Li, Hui Yuan, Weihao Zhu, Hao Dong, Zilong Zhu, Siyu Computer Vision and Pattern Recognition Video face restoration faces a critical challenge in maintaining temporal consistency while recovering fine facial details from degraded inputs. This paper presents a novel approach that extends Vector-Quantized Variational Autoencoders (VQ-VAEs), pretrained on static high-quality portraits, into a video restoration framework through variational latent space modeling. Our key innovation lies in reformulating discrete codebook representations as Dirichlet-distributed continuous variables, enabling probabilistic transitions between facial features across frames. A spatio-temporal Transformer architecture jointly models inter-frame dependencies and predicts latent distributions, while a Laplacian-constrained reconstruction loss combined with perceptual (LPIPS) regularization enhances both pixel accuracy and visual quality. Comprehensive evaluations on blind face restoration, video inpainting, and facial colorization tasks demonstrate state-of-the-art performance. This work establishes an effective paradigm for adapting intensive image priors, pretrained on high-quality images, to video restoration while addressing the critical challenge of flicker artifacts. The source code has been open-sourced and is available at https://github.com/fudan-generative-vision/DicFace. |
| title | DicFace: Dirichlet-Constrained Variational Codebook Learning for Temporally Coherent Video Face Restoration |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.13355 |