InstructBrush: Learning Attention-based Instruction Optimization for Image Editing
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911816814166016 |
|---|---|
| author | Zhao, Ruoyu Fan, Qingnan Kou, Fei Qin, Shuai Gu, Hong Wu, Wei Xu, Pengcheng Zhu, Mingrui Wang, Nannan Gao, Xinbo |
| author_facet | Zhao, Ruoyu Fan, Qingnan Kou, Fei Qin, Shuai Gu, Hong Wu, Wei Xu, Pengcheng Zhu, Mingrui Wang, Nannan Gao, Xinbo |
| contents | In recent years, instruction-based image editing methods have garnered significant attention in image editing. However, despite encompassing a wide range of editing priors, these methods are helpless when handling editing tasks that are challenging to accurately describe through language. We propose InstructBrush, an inversion method for instruction-based image editing methods to bridge this gap. It extracts editing effects from exemplar image pairs as editing instructions, which are further applied for image editing. Two key techniques are introduced into InstructBrush, Attention-based Instruction Optimization and Transformation-oriented Instruction Initialization, to address the limitations of the previous method in terms of inversion effects and instruction generalization. To explore the ability of instruction inversion methods to guide image editing in open scenarios, we establish a TransformationOriented Paired Benchmark (TOP-Bench), which contains a rich set of scenes and editing types. The creation of this benchmark paves the way for further exploration of instruction inversion. Quantitatively and qualitatively, our approach achieves superior performance in editing and is more semantically consistent with the target editing effects. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_18660 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | InstructBrush: Learning Attention-based Instruction Optimization for Image Editing Zhao, Ruoyu Fan, Qingnan Kou, Fei Qin, Shuai Gu, Hong Wu, Wei Xu, Pengcheng Zhu, Mingrui Wang, Nannan Gao, Xinbo Graphics Computer Vision and Pattern Recognition In recent years, instruction-based image editing methods have garnered significant attention in image editing. However, despite encompassing a wide range of editing priors, these methods are helpless when handling editing tasks that are challenging to accurately describe through language. We propose InstructBrush, an inversion method for instruction-based image editing methods to bridge this gap. It extracts editing effects from exemplar image pairs as editing instructions, which are further applied for image editing. Two key techniques are introduced into InstructBrush, Attention-based Instruction Optimization and Transformation-oriented Instruction Initialization, to address the limitations of the previous method in terms of inversion effects and instruction generalization. To explore the ability of instruction inversion methods to guide image editing in open scenarios, we establish a TransformationOriented Paired Benchmark (TOP-Bench), which contains a rich set of scenes and editing types. The creation of this benchmark paves the way for further exploration of instruction inversion. Quantitatively and qualitatively, our approach achieves superior performance in editing and is more semantically consistent with the target editing effects. |
| title | InstructBrush: Learning Attention-based Instruction Optimization for Image Editing |
| topic | Graphics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.18660 |