MegaFusion: Extend Diffusion Models towards Higher-resolution Image Generation without Further Tuning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917839839952896 |
|---|---|
| author | Wu, Haoning Shen, Shaocheng Hu, Qiang Zhang, Xiaoyun Zhang, Ya Wang, Yanfeng |
| author_facet | Wu, Haoning Shen, Shaocheng Hu, Qiang Zhang, Xiaoyun Zhang, Ya Wang, Yanfeng |
| contents | Diffusion models have emerged as frontrunners in text-to-image generation, but their fixed image resolution during training often leads to challenges in high-resolution image generation, such as semantic deviations and object replication. This paper introduces MegaFusion, a novel approach that extends existing diffusion-based text-to-image models towards efficient higher-resolution generation without additional fine-tuning or adaptation. Specifically, we employ an innovative truncate and relay strategy to bridge the denoising processes across different resolutions, allowing for high-resolution image generation in a coarse-to-fine manner. Moreover, by integrating dilated convolutions and noise re-scheduling, we further adapt the model's priors for higher resolution. The versatility and efficacy of MegaFusion make it universally applicable to both latent-space and pixel-space diffusion models, along with other derivative models. Extensive experiments confirm that MegaFusion significantly boosts the capability of existing models to produce images of megapixels and various aspect ratios, while only requiring about 40% of the original computational cost. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_11001 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MegaFusion: Extend Diffusion Models towards Higher-resolution Image Generation without Further Tuning Wu, Haoning Shen, Shaocheng Hu, Qiang Zhang, Xiaoyun Zhang, Ya Wang, Yanfeng Computer Vision and Pattern Recognition Diffusion models have emerged as frontrunners in text-to-image generation, but their fixed image resolution during training often leads to challenges in high-resolution image generation, such as semantic deviations and object replication. This paper introduces MegaFusion, a novel approach that extends existing diffusion-based text-to-image models towards efficient higher-resolution generation without additional fine-tuning or adaptation. Specifically, we employ an innovative truncate and relay strategy to bridge the denoising processes across different resolutions, allowing for high-resolution image generation in a coarse-to-fine manner. Moreover, by integrating dilated convolutions and noise re-scheduling, we further adapt the model's priors for higher resolution. The versatility and efficacy of MegaFusion make it universally applicable to both latent-space and pixel-space diffusion models, along with other derivative models. Extensive experiments confirm that MegaFusion significantly boosts the capability of existing models to produce images of megapixels and various aspect ratios, while only requiring about 40% of the original computational cost. |
| title | MegaFusion: Extend Diffusion Models towards Higher-resolution Image Generation without Further Tuning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.11001 |