ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motions
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918054449905664 |
|---|---|
| author | Chang, Di Cao, Mingdeng Shi, Yichun Liu, Bo Cai, Shengqu Zhou, Shijie Huang, Weilin Wetzstein, Gordon Soleymani, Mohammad Wang, Peng |
| author_facet | Chang, Di Cao, Mingdeng Shi, Yichun Liu, Bo Cai, Shengqu Zhou, Shijie Huang, Weilin Wetzstein, Gordon Soleymani, Mohammad Wang, Peng |
| contents | Editing images with instructions to reflect non-rigid motions, camera viewpoint shifts, object deformations, human articulations, and complex interactions, poses a challenging yet underexplored problem in computer vision. Existing approaches and datasets predominantly focus on static scenes or rigid transformations, limiting their capacity to handle expressive edits involving dynamic motion. To address this gap, we introduce ByteMorph, a comprehensive framework for instruction-based image editing with an emphasis on non-rigid motions. ByteMorph comprises a large-scale dataset, ByteMorph-6M, and a strong baseline model built upon the Diffusion Transformer (DiT), named ByteMorpher. ByteMorph-6M includes over 6 million high-resolution image editing pairs for training, along with a carefully curated evaluation benchmark ByteMorph-Bench. Both capture a wide variety of non-rigid motion types across diverse environments, human figures, and object categories. The dataset is constructed using motion-guided data generation, layered compositing techniques, and automated captioning to ensure diversity, realism, and semantic coherence. We further conduct a comprehensive evaluation of recent instruction-based image editing methods from both academic and commercial domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_03107 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motions Chang, Di Cao, Mingdeng Shi, Yichun Liu, Bo Cai, Shengqu Zhou, Shijie Huang, Weilin Wetzstein, Gordon Soleymani, Mohammad Wang, Peng Computer Vision and Pattern Recognition Editing images with instructions to reflect non-rigid motions, camera viewpoint shifts, object deformations, human articulations, and complex interactions, poses a challenging yet underexplored problem in computer vision. Existing approaches and datasets predominantly focus on static scenes or rigid transformations, limiting their capacity to handle expressive edits involving dynamic motion. To address this gap, we introduce ByteMorph, a comprehensive framework for instruction-based image editing with an emphasis on non-rigid motions. ByteMorph comprises a large-scale dataset, ByteMorph-6M, and a strong baseline model built upon the Diffusion Transformer (DiT), named ByteMorpher. ByteMorph-6M includes over 6 million high-resolution image editing pairs for training, along with a carefully curated evaluation benchmark ByteMorph-Bench. Both capture a wide variety of non-rigid motion types across diverse environments, human figures, and object categories. The dataset is constructed using motion-guided data generation, layered compositing techniques, and automated captioning to ensure diversity, realism, and semantic coherence. We further conduct a comprehensive evaluation of recent instruction-based image editing methods from both academic and commercial domains. |
| title | ByteMorph: Benchmarking Instruction-Guided Image Editing with Non-Rigid Motions |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.03107 |