Omni IIE Bench: Benchmarking the Practical Capabilities of Image Editing Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915870752636928 |
|---|---|
| author | Yang, Yujia Wang, Yuanxiang Guan, Zhenyu Yang, Tiankun Bao, Chenxi Jin, Haopeng Luo, Jinwen Zuo, Xinyu Duan, Lisheng Liang, Haijin Ma, Jin Wang, Xinming Tao, Ruiwen Yi, Hongzhu |
| author_facet | Yang, Yujia Wang, Yuanxiang Guan, Zhenyu Yang, Tiankun Bao, Chenxi Jin, Haopeng Luo, Jinwen Zuo, Xinyu Duan, Lisheng Liang, Haijin Ma, Jin Wang, Xinming Tao, Ruiwen Yi, Hongzhu |
| contents | While Instruction-based Image Editing (IIE) has achieved significant progress, existing benchmarks pursue task breadth via mixed evaluations. This paradigm obscures a critical failure mode crucial in professional applications: the inconsistent performance of models across tasks of varying semantic scales. To address this gap, we introduce Omni IIE Bench, a high-quality, human-annotated benchmark specifically designed to diagnose the editing consistency of IIE models in practical application scenarios. Omni IIE Bench features an innovative dual-track diagnostic design: (1) Single-turn Consistency, comprising shared-context task pairs of attribute modification and entity replacement; and (2) Multi-turn Coordination, involving continuous dialogue tasks that traverse semantic scales. The benchmark is constructed via an exceptionally rigorous multi-stage human filtering process, incorporating a quality standard enforced by computer vision graduate students and an industry relevance review conducted by professional designers. We perform a comprehensive evaluation of 8 mainstream IIE models using Omni IIE Bench. Our analysis quantifies, for the first time, a prevalent performance gap: nearly all models exhibit a significant performance degradation when transitioning from low-semantic-scale to high-semantic-scale tasks. Omni IIE Bench provides critical diagnostic tools and insights for the development of next-generation, more reliable, and stable IIE models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_16944 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Omni IIE Bench: Benchmarking the Practical Capabilities of Image Editing Models Yang, Yujia Wang, Yuanxiang Guan, Zhenyu Yang, Tiankun Bao, Chenxi Jin, Haopeng Luo, Jinwen Zuo, Xinyu Duan, Lisheng Liang, Haijin Ma, Jin Wang, Xinming Tao, Ruiwen Yi, Hongzhu Computer Vision and Pattern Recognition Artificial Intelligence While Instruction-based Image Editing (IIE) has achieved significant progress, existing benchmarks pursue task breadth via mixed evaluations. This paradigm obscures a critical failure mode crucial in professional applications: the inconsistent performance of models across tasks of varying semantic scales. To address this gap, we introduce Omni IIE Bench, a high-quality, human-annotated benchmark specifically designed to diagnose the editing consistency of IIE models in practical application scenarios. Omni IIE Bench features an innovative dual-track diagnostic design: (1) Single-turn Consistency, comprising shared-context task pairs of attribute modification and entity replacement; and (2) Multi-turn Coordination, involving continuous dialogue tasks that traverse semantic scales. The benchmark is constructed via an exceptionally rigorous multi-stage human filtering process, incorporating a quality standard enforced by computer vision graduate students and an industry relevance review conducted by professional designers. We perform a comprehensive evaluation of 8 mainstream IIE models using Omni IIE Bench. Our analysis quantifies, for the first time, a prevalent performance gap: nearly all models exhibit a significant performance degradation when transitioning from low-semantic-scale to high-semantic-scale tasks. Omni IIE Bench provides critical diagnostic tools and insights for the development of next-generation, more reliable, and stable IIE models. |
| title | Omni IIE Bench: Benchmarking the Practical Capabilities of Image Editing Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2603.16944 |