FreeCompose: Generic Zero-Shot Image Composition with Diffusion Prior
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916314095812608 |
|---|---|
| author | Chen, Zhekai Wang, Wen Yang, Zhen Yuan, Zeqing Chen, Hao Shen, Chunhua |
| author_facet | Chen, Zhekai Wang, Wen Yang, Zhen Yuan, Zeqing Chen, Hao Shen, Chunhua |
| contents | We offer a novel approach to image composition, which integrates multiple input images into a single, coherent image. Rather than concentrating on specific use cases such as appearance editing (image harmonization) or semantic editing (semantic image composition), we showcase the potential of utilizing the powerful generative prior inherent in large-scale pre-trained diffusion models to accomplish generic image composition applicable to both scenarios. We observe that the pre-trained diffusion models automatically identify simple copy-paste boundary areas as low-density regions during denoising. Building on this insight, we propose to optimize the composed image towards high-density regions guided by the diffusion prior. In addition, we introduce a novel maskguided loss to further enable flexible semantic image composition. Extensive experiments validate the superiority of our approach in achieving generic zero-shot image composition. Additionally, our approach shows promising potential in various tasks, such as object removal and multiconcept customization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_04947 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FreeCompose: Generic Zero-Shot Image Composition with Diffusion Prior Chen, Zhekai Wang, Wen Yang, Zhen Yuan, Zeqing Chen, Hao Shen, Chunhua Computer Vision and Pattern Recognition We offer a novel approach to image composition, which integrates multiple input images into a single, coherent image. Rather than concentrating on specific use cases such as appearance editing (image harmonization) or semantic editing (semantic image composition), we showcase the potential of utilizing the powerful generative prior inherent in large-scale pre-trained diffusion models to accomplish generic image composition applicable to both scenarios. We observe that the pre-trained diffusion models automatically identify simple copy-paste boundary areas as low-density regions during denoising. Building on this insight, we propose to optimize the composed image towards high-density regions guided by the diffusion prior. In addition, we introduce a novel maskguided loss to further enable flexible semantic image composition. Extensive experiments validate the superiority of our approach in achieving generic zero-shot image composition. Additionally, our approach shows promising potential in various tasks, such as object removal and multiconcept customization. |
| title | FreeCompose: Generic Zero-Shot Image Composition with Diffusion Prior |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.04947 |