Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2411.18499 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915218241617920 |
|---|---|
| author | Zhou, Pengfei Peng, Xiaopeng Song, Jiajun Li, Chuanhao Xu, Zhaopan Yang, Yue Guo, Ziyao Zhang, Hao Lin, Yuqi He, Yefei Zhao, Lirui Liu, Shuo Li, Tianhua Xie, Yuxuan Chang, Xiaojun Qiao, Yu Shao, Wenqi Zhang, Kaipeng |
| author_facet | Zhou, Pengfei Peng, Xiaopeng Song, Jiajun Li, Chuanhao Xu, Zhaopan Yang, Yue Guo, Ziyao Zhang, Hao Lin, Yuqi He, Yefei Zhao, Lirui Liu, Shuo Li, Tianhua Xie, Yuxuan Chang, Xiaojun Qiao, Yu Shao, Wenqi Zhang, Kaipeng |
| contents | Multimodal Large Language Models (MLLMs) have made significant strides in visual understanding and generation tasks. However, generating interleaved image-text content remains a challenge, which requires integrated multimodal understanding and generation abilities. While the progress in unified models offers new solutions, existing benchmarks are insufficient for evaluating these methods due to limitations in data size and diversity. To bridge this gap, we introduce OpenING, a comprehensive benchmark comprising 5,400 high-quality human-annotated instances across 56 real-world tasks. OpenING covers diverse daily scenarios such as travel guide, design, and brainstorming, offering a robust platform for challenging interleaved generation methods. In addition, we present IntJudge, a judge model for evaluating open-ended multimodal generation methods. Trained with a novel data pipeline, our IntJudge achieves an agreement rate of 82.42% with human judgments, outperforming GPT-based evaluators by 11.34%. Extensive experiments on OpenING reveal that current interleaved generation methods still have substantial room for improvement. Key findings on interleaved image-text generation are further presented to guide the development of next-generation models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_18499 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation Zhou, Pengfei Peng, Xiaopeng Song, Jiajun Li, Chuanhao Xu, Zhaopan Yang, Yue Guo, Ziyao Zhang, Hao Lin, Yuqi He, Yefei Zhao, Lirui Liu, Shuo Li, Tianhua Xie, Yuxuan Chang, Xiaojun Qiao, Yu Shao, Wenqi Zhang, Kaipeng Computer Vision and Pattern Recognition Multimodal Large Language Models (MLLMs) have made significant strides in visual understanding and generation tasks. However, generating interleaved image-text content remains a challenge, which requires integrated multimodal understanding and generation abilities. While the progress in unified models offers new solutions, existing benchmarks are insufficient for evaluating these methods due to limitations in data size and diversity. To bridge this gap, we introduce OpenING, a comprehensive benchmark comprising 5,400 high-quality human-annotated instances across 56 real-world tasks. OpenING covers diverse daily scenarios such as travel guide, design, and brainstorming, offering a robust platform for challenging interleaved generation methods. In addition, we present IntJudge, a judge model for evaluating open-ended multimodal generation methods. Trained with a novel data pipeline, our IntJudge achieves an agreement rate of 82.42% with human judgments, outperforming GPT-based evaluators by 11.34%. Extensive experiments on OpenING reveal that current interleaved generation methods still have substantial room for improvement. Key findings on interleaved image-text generation are further presented to guide the development of next-generation models. |
| title | OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.18499 |