Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.12883 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909791229575168 |
|---|---|
| author | Jia, Qifei Liu, Yu Chai, Yajie Yao, Xintong Lu, Qiming Zhang, Yasen Shi, Runyu Huang, Ying Zhang, Guoquan |
| author_facet | Jia, Qifei Liu, Yu Chai, Yajie Yao, Xintong Lu, Qiming Zhang, Yasen Shi, Runyu Huang, Ying Zhang, Guoquan |
| contents | Instruction-based image editing has garnered significant attention due to its direct interaction with users. However, real-world user instructions are immensely diverse, and existing methods often fail to generalize effectively to instructions outside their training domain, limiting their practical application. To address this, we propose Lego-Edit, which leverages the generalization capability of Multi-modal Large Language Model (MLLM) to organize a suite of model-level editing tools to tackle this challenge. Lego-Edit incorporates two key designs: (1) a model-level toolkit comprising diverse models efficiently trained on limited data and several image manipulation functions, enabling fine-grained composition of editing actions by the MLLM; and (2) a three-stage progressive reinforcement learning approach that uses feedback on unannotated, open-domain instructions to train the MLLM, equipping it with generalized reasoning capabilities for handling real-world instructions. Experiments demonstrate that Lego-Edit achieves state-of-the-art performance on GEdit-Bench and ImgBench. It exhibits robust reasoning capabilities for open-domain instructions and can utilize newly introduced editing tools without additional fine-tuning.
Code is available: https://github.com/xiaomi-research/lego-edit. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_12883 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Lego-Edit: A General Image Editing Framework with Model-Level Bricks and MLLM Builder Jia, Qifei Liu, Yu Chai, Yajie Yao, Xintong Lu, Qiming Zhang, Yasen Shi, Runyu Huang, Ying Zhang, Guoquan Computer Vision and Pattern Recognition Instruction-based image editing has garnered significant attention due to its direct interaction with users. However, real-world user instructions are immensely diverse, and existing methods often fail to generalize effectively to instructions outside their training domain, limiting their practical application. To address this, we propose Lego-Edit, which leverages the generalization capability of Multi-modal Large Language Model (MLLM) to organize a suite of model-level editing tools to tackle this challenge. Lego-Edit incorporates two key designs: (1) a model-level toolkit comprising diverse models efficiently trained on limited data and several image manipulation functions, enabling fine-grained composition of editing actions by the MLLM; and (2) a three-stage progressive reinforcement learning approach that uses feedback on unannotated, open-domain instructions to train the MLLM, equipping it with generalized reasoning capabilities for handling real-world instructions. Experiments demonstrate that Lego-Edit achieves state-of-the-art performance on GEdit-Bench and ImgBench. It exhibits robust reasoning capabilities for open-domain instructions and can utilize newly introduced editing tools without additional fine-tuning. Code is available: https://github.com/xiaomi-research/lego-edit. |
| title | Lego-Edit: A General Image Editing Framework with Model-Level Bricks and MLLM Builder |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2509.12883 |