ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image Restoration
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909575662272512 |
|---|---|
| author | Yu, Yongsheng Zheng, Haitian Zhang, Zhifei Zhang, Jianming Zhou, Yuqian Barnes, Connelly Liu, Yuchen Xiong, Wei Lin, Zhe Luo, Jiebo |
| author_facet | Yu, Yongsheng Zheng, Haitian Zhang, Zhifei Zhang, Jianming Zhou, Yuqian Barnes, Connelly Liu, Yuchen Xiong, Wei Lin, Zhe Luo, Jiebo |
| contents | Recent progress in generative models has significantly improved image restoration capabilities, particularly through powerful diffusion models that offer remarkable recovery of semantic details and local fidelity. However, deploying these models at ultra-high resolutions faces a critical trade-off between quality and efficiency due to the computational demands of long-range attention mechanisms. To address this, we introduce ZipIR, a novel framework that enhances efficiency, scalability, and long-range modeling for high-res image restoration. ZipIR employs a highly compressed latent representation that compresses image 32x, effectively reducing the number of spatial tokens, and enabling the use of high-capacity models like the Diffusion Transformer (DiT). Toward this goal, we propose a Latent Pyramid VAE (LP-VAE) design that structures the latent space into sub-bands to ease diffusion training. Trained on full images up to 2K resolution, ZipIR surpasses existing diffusion-based methods, offering unmatched speed and quality in restoring high-resolution images from severely degraded inputs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_08591 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image Restoration Yu, Yongsheng Zheng, Haitian Zhang, Zhifei Zhang, Jianming Zhou, Yuqian Barnes, Connelly Liu, Yuchen Xiong, Wei Lin, Zhe Luo, Jiebo Computer Vision and Pattern Recognition Recent progress in generative models has significantly improved image restoration capabilities, particularly through powerful diffusion models that offer remarkable recovery of semantic details and local fidelity. However, deploying these models at ultra-high resolutions faces a critical trade-off between quality and efficiency due to the computational demands of long-range attention mechanisms. To address this, we introduce ZipIR, a novel framework that enhances efficiency, scalability, and long-range modeling for high-res image restoration. ZipIR employs a highly compressed latent representation that compresses image 32x, effectively reducing the number of spatial tokens, and enabling the use of high-capacity models like the Diffusion Transformer (DiT). Toward this goal, we propose a Latent Pyramid VAE (LP-VAE) design that structures the latent space into sub-bands to ease diffusion training. Trained on full images up to 2K resolution, ZipIR surpasses existing diffusion-based methods, offering unmatched speed and quality in restoring high-resolution images from severely degraded inputs. |
| title | ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image Restoration |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.08591 |