ReEdit: Multimodal Exemplar-Based Image Editing with Diffusion Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915007657148416 |
|---|---|
| author | Srivastava, Ashutosh Menta, Tarun Ram Java, Abhinav Jadhav, Avadhoot Singh, Silky Jandial, Surgan Krishnamurthy, Balaji |
| author_facet | Srivastava, Ashutosh Menta, Tarun Ram Java, Abhinav Jadhav, Avadhoot Singh, Silky Jandial, Surgan Krishnamurthy, Balaji |
| contents | Modern Text-to-Image (T2I) Diffusion models have revolutionized image editing by enabling the generation of high-quality photorealistic images. While the de facto method for performing edits with T2I models is through text instructions, this approach non-trivial due to the complex many-to-many mapping between natural language and images. In this work, we address exemplar-based image editing -- the task of transferring an edit from an exemplar pair to a content image(s). We propose ReEdit, a modular and efficient end-to-end framework that captures edits in both text and image modalities while ensuring the fidelity of the edited image. We validate the effectiveness of ReEdit through extensive comparisons with state-of-the-art baselines and sensitivity analyses of key design choices. Our results demonstrate that ReEdit consistently outperforms contemporary approaches both qualitatively and quantitatively. Additionally, ReEdit boasts high practical applicability, as it does not require any task-specific optimization and is four times faster than the next best baseline. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_03982 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ReEdit: Multimodal Exemplar-Based Image Editing with Diffusion Models Srivastava, Ashutosh Menta, Tarun Ram Java, Abhinav Jadhav, Avadhoot Singh, Silky Jandial, Surgan Krishnamurthy, Balaji Computer Vision and Pattern Recognition Modern Text-to-Image (T2I) Diffusion models have revolutionized image editing by enabling the generation of high-quality photorealistic images. While the de facto method for performing edits with T2I models is through text instructions, this approach non-trivial due to the complex many-to-many mapping between natural language and images. In this work, we address exemplar-based image editing -- the task of transferring an edit from an exemplar pair to a content image(s). We propose ReEdit, a modular and efficient end-to-end framework that captures edits in both text and image modalities while ensuring the fidelity of the edited image. We validate the effectiveness of ReEdit through extensive comparisons with state-of-the-art baselines and sensitivity analyses of key design choices. Our results demonstrate that ReEdit consistently outperforms contemporary approaches both qualitatively and quantitatively. Additionally, ReEdit boasts high practical applicability, as it does not require any task-specific optimization and is four times faster than the next best baseline. |
| title | ReEdit: Multimodal Exemplar-Based Image Editing with Diffusion Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.03982 |