Textual and Visual Prompt Fusion for Image Editing via Step-Wise Alignment
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917883377876992 |
|---|---|
| author | Feng, Zhanbo Ling, Zenan Lu, Xinyu Gong, Ci Zhou, Feng Bao, Wugedele Li, Jie Yang, Fan Qiu, Robert C. |
| author_facet | Feng, Zhanbo Ling, Zenan Lu, Xinyu Gong, Ci Zhou, Feng Bao, Wugedele Li, Jie Yang, Fan Qiu, Robert C. |
| contents | The use of denoising diffusion models is becoming increasingly popular in the field of image editing. However, current approaches often rely on either image-guided methods, which provide a visual reference but lack control over semantic consistency, or text-guided methods, which ensure alignment with the text guidance but compromise visual quality. To resolve this issue, we propose a framework that integrates a fusion of generated visual references and text guidance into the semantic latent space of a \textit{frozen} pre-trained diffusion model. Using only a tiny neural network, our framework provides control over diverse content and attributes, driven intuitively by the text prompt. Compared to state-of-the-art methods, the framework generates images of higher quality while providing realistic editing effects across various benchmark datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_15854 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Textual and Visual Prompt Fusion for Image Editing via Step-Wise Alignment Feng, Zhanbo Ling, Zenan Lu, Xinyu Gong, Ci Zhou, Feng Bao, Wugedele Li, Jie Yang, Fan Qiu, Robert C. Computer Vision and Pattern Recognition Artificial Intelligence The use of denoising diffusion models is becoming increasingly popular in the field of image editing. However, current approaches often rely on either image-guided methods, which provide a visual reference but lack control over semantic consistency, or text-guided methods, which ensure alignment with the text guidance but compromise visual quality. To resolve this issue, we propose a framework that integrates a fusion of generated visual references and text guidance into the semantic latent space of a \textit{frozen} pre-trained diffusion model. Using only a tiny neural network, our framework provides control over diverse content and attributes, driven intuitively by the text prompt. Compared to state-of-the-art methods, the framework generates images of higher quality while providing realistic editing effects across various benchmark datasets. |
| title | Textual and Visual Prompt Fusion for Image Editing via Step-Wise Alignment |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2308.15854 |