Editable Image Elements for Controllable Synthesis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917649101881344 |
|---|---|
| author | Mu, Jiteng Gharbi, Michaël Zhang, Richard Shechtman, Eli Vasconcelos, Nuno Wang, Xiaolong Park, Taesung |
| author_facet | Mu, Jiteng Gharbi, Michaël Zhang, Richard Shechtman, Eli Vasconcelos, Nuno Wang, Xiaolong Park, Taesung |
| contents | Diffusion models have made significant advances in text-guided synthesis tasks. However, editing user-provided images remains challenging, as the high dimensional noise input space of diffusion models is not naturally suited for image inversion or spatial editing. In this work, we propose an image representation that promotes spatial editing of input images using a diffusion model. Concretely, we learn to encode an input into "image elements" that can faithfully reconstruct an input image. These elements can be intuitively edited by a user, and are decoded by a diffusion model into realistic images. We show the effectiveness of our representation on various image editing tasks, such as object resizing, rearrangement, dragging, de-occlusion, removal, variation, and image composition. Project page: https://jitengmu.github.io/Editable_Image_Elements/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_16029 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Editable Image Elements for Controllable Synthesis Mu, Jiteng Gharbi, Michaël Zhang, Richard Shechtman, Eli Vasconcelos, Nuno Wang, Xiaolong Park, Taesung Computer Vision and Pattern Recognition Diffusion models have made significant advances in text-guided synthesis tasks. However, editing user-provided images remains challenging, as the high dimensional noise input space of diffusion models is not naturally suited for image inversion or spatial editing. In this work, we propose an image representation that promotes spatial editing of input images using a diffusion model. Concretely, we learn to encode an input into "image elements" that can faithfully reconstruct an input image. These elements can be intuitively edited by a user, and are decoded by a diffusion model into realistic images. We show the effectiveness of our representation on various image editing tasks, such as object resizing, rearrangement, dragging, de-occlusion, removal, variation, and image composition. Project page: https://jitengmu.github.io/Editable_Image_Elements/ |
| title | Editable Image Elements for Controllable Synthesis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.16029 |