VINCIE: Unlocking In-context Image Editing from Video
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915825031577600 |
|---|---|
| author | Qu, Leigang Cheng, Feng Yang, Ziyan Zhao, Qi Lin, Shanchuan Shi, Yichun Li, Yicong Wang, Wenjie Chua, Tat-Seng Jiang, Lu |
| author_facet | Qu, Leigang Cheng, Feng Yang, Ziyan Zhao, Qi Lin, Shanchuan Shi, Yichun Li, Yicong Wang, Wenjie Chua, Tat-Seng Jiang, Lu |
| contents | In-context image editing aims to modify images based on a contextual sequence comprising text and previously generated images. Existing methods typically depend on task-specific pipelines and expert models (e.g., segmentation and inpainting) to curate training data. In this work, we explore whether an in-context image editing model can be learned directly from videos. We introduce a scalable approach to annotate videos as interleaved multimodal sequences. To effectively learn from this data, we design a block-causal diffusion transformer trained on three proxy tasks: next-image prediction, current segmentation prediction, and next-segmentation prediction. Additionally, we propose a novel multi-turn image editing benchmark to advance research in this area. Extensive experiments demonstrate that our model exhibits strong in-context image editing capabilities and achieves state-of-the-art results on two multi-turn image editing benchmarks. Despite being trained exclusively on videos, our model also shows promising abilities in multi-concept composition, story generation, and chain-of-editing applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_10941 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VINCIE: Unlocking In-context Image Editing from Video Qu, Leigang Cheng, Feng Yang, Ziyan Zhao, Qi Lin, Shanchuan Shi, Yichun Li, Yicong Wang, Wenjie Chua, Tat-Seng Jiang, Lu Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning Multimedia In-context image editing aims to modify images based on a contextual sequence comprising text and previously generated images. Existing methods typically depend on task-specific pipelines and expert models (e.g., segmentation and inpainting) to curate training data. In this work, we explore whether an in-context image editing model can be learned directly from videos. We introduce a scalable approach to annotate videos as interleaved multimodal sequences. To effectively learn from this data, we design a block-causal diffusion transformer trained on three proxy tasks: next-image prediction, current segmentation prediction, and next-segmentation prediction. Additionally, we propose a novel multi-turn image editing benchmark to advance research in this area. Extensive experiments demonstrate that our model exhibits strong in-context image editing capabilities and achieves state-of-the-art results on two multi-turn image editing benchmarks. Despite being trained exclusively on videos, our model also shows promising abilities in multi-concept composition, story generation, and chain-of-editing applications. |
| title | VINCIE: Unlocking In-context Image Editing from Video |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning Multimedia |
| url | https://arxiv.org/abs/2506.10941 |