Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.12963 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911803722694656 |
|---|---|
| author | Huang, Linjiang Fang, Rongyao Zhang, Aiping Song, Guanglu Liu, Si Liu, Yu Li, Hongsheng |
| author_facet | Huang, Linjiang Fang, Rongyao Zhang, Aiping Song, Guanglu Liu, Si Liu, Yu Li, Hongsheng |
| contents | In this study, we delve into the generation of high-resolution images from pre-trained diffusion models, addressing persistent challenges, such as repetitive patterns and structural distortions, that emerge when models are applied beyond their trained resolutions. To address this issue, we introduce an innovative, training-free approach FouriScale from the perspective of frequency domain analysis. We replace the original convolutional layers in pre-trained diffusion models by incorporating a dilation technique along with a low-pass operation, intending to achieve structural consistency and scale consistency across resolutions, respectively. Further enhanced by a padding-then-crop strategy, our method can flexibly handle text-to-image generation of various aspect ratios. By using the FouriScale as guidance, our method successfully balances the structural integrity and fidelity of generated images, achieving an astonishing capacity of arbitrary-size, high-resolution, and high-quality generation. With its simplicity and compatibility, our method can provide valuable insights for future explorations into the synthesis of ultra-high-resolution images. The code will be released at https://github.com/LeonHLJ/FouriScale. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_12963 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesis Huang, Linjiang Fang, Rongyao Zhang, Aiping Song, Guanglu Liu, Si Liu, Yu Li, Hongsheng Computer Vision and Pattern Recognition In this study, we delve into the generation of high-resolution images from pre-trained diffusion models, addressing persistent challenges, such as repetitive patterns and structural distortions, that emerge when models are applied beyond their trained resolutions. To address this issue, we introduce an innovative, training-free approach FouriScale from the perspective of frequency domain analysis. We replace the original convolutional layers in pre-trained diffusion models by incorporating a dilation technique along with a low-pass operation, intending to achieve structural consistency and scale consistency across resolutions, respectively. Further enhanced by a padding-then-crop strategy, our method can flexibly handle text-to-image generation of various aspect ratios. By using the FouriScale as guidance, our method successfully balances the structural integrity and fidelity of generated images, achieving an astonishing capacity of arbitrary-size, high-resolution, and high-quality generation. With its simplicity and compatibility, our method can provide valuable insights for future explorations into the synthesis of ultra-high-resolution images. The code will be released at https://github.com/LeonHLJ/FouriScale. |
| title | FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.12963 |