Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.13081 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915069095313408 |
|---|---|
| author | Bodur, Rumeysa Bhattarai, Binod Kim, Tae-Kyun |
| author_facet | Bodur, Rumeysa Bhattarai, Binod Kim, Tae-Kyun |
| contents | Text-guided image editing finds applications in various creative and practical fields. While recent studies in image generation have advanced the field, they often struggle with the dual challenges of coherent image transformation and context preservation. In response, our work introduces prompt augmentation, a method amplifying a single input prompt into several target prompts, strengthening textual context and enabling localised image editing. Specifically, we use the augmented prompts to delineate the intended manipulation area. We propose a Contrastive Loss tailored to driving effective image editing by displacing edited areas and drawing preserved regions closer. Acknowledging the continuous nature of image manipulations, we further refine our approach by incorporating the similarity concept, creating a Soft Contrastive Loss. The new losses are incorporated to the diffusion model, demonstrating improved or competitive image editing results on public datasets and generated images over state-of-the-art approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_13081 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Prompt Augmentation for Self-supervised Text-guided Image Manipulation Bodur, Rumeysa Bhattarai, Binod Kim, Tae-Kyun Computer Vision and Pattern Recognition Text-guided image editing finds applications in various creative and practical fields. While recent studies in image generation have advanced the field, they often struggle with the dual challenges of coherent image transformation and context preservation. In response, our work introduces prompt augmentation, a method amplifying a single input prompt into several target prompts, strengthening textual context and enabling localised image editing. Specifically, we use the augmented prompts to delineate the intended manipulation area. We propose a Contrastive Loss tailored to driving effective image editing by displacing edited areas and drawing preserved regions closer. Acknowledging the continuous nature of image manipulations, we further refine our approach by incorporating the similarity concept, creating a Soft Contrastive Loss. The new losses are incorporated to the diffusion model, demonstrating improved or competitive image editing results on public datasets and generated images over state-of-the-art approaches. |
| title | Prompt Augmentation for Self-supervised Text-guided Image Manipulation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.13081 |