Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911602303827968 |
|---|---|
| author | Zou, Kai Huang, Ziqi Dong, Yuhao Tian, Shulin Zheng, Dian Liu, Hongbo He, Jingwen Liu, Bin Qiao, Yu Liu, Ziwei |
| author_facet | Zou, Kai Huang, Ziqi Dong, Yuhao Tian, Shulin Zheng, Dian Liu, Hongbo He, Jingwen Liu, Bin Qiao, Yu Liu, Ziwei |
| contents | Unified multimodal models aim to jointly enable visual understanding and generation, yet current benchmarks rarely examine their true integration. Existing evaluations either treat the two abilities in isolation or overlook tasks that inherently couple them. To address this gap, we present Uni-MMMU, a comprehensive and discipline-aware benchmark that systematically unfolds the bidirectional synergy between generation and understanding across eight reasoning-centric domains, including science, coding, mathematics, and puzzles. Each task is bidirectionally coupled, demanding models to (i) leverage conceptual understanding to guide precise visual synthesis, or (ii) utilize generation as a cognitive scaffold for analytical reasoning. Uni-MMMU incorporates verifiable intermediate reasoning steps, unique ground truths, and a reproducible scoring protocol for both textual and visual outputs. Through extensive evaluation of state-of-the-art unified, generation-only, and understanding-only models, we reveal substantial performance disparities and cross-modal dependencies, offering new insights into when and how these abilities reinforce one another, and establishing a reliable foundation for advancing unified models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_13759 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark Zou, Kai Huang, Ziqi Dong, Yuhao Tian, Shulin Zheng, Dian Liu, Hongbo He, Jingwen Liu, Bin Qiao, Yu Liu, Ziwei Computer Vision and Pattern Recognition Unified multimodal models aim to jointly enable visual understanding and generation, yet current benchmarks rarely examine their true integration. Existing evaluations either treat the two abilities in isolation or overlook tasks that inherently couple them. To address this gap, we present Uni-MMMU, a comprehensive and discipline-aware benchmark that systematically unfolds the bidirectional synergy between generation and understanding across eight reasoning-centric domains, including science, coding, mathematics, and puzzles. Each task is bidirectionally coupled, demanding models to (i) leverage conceptual understanding to guide precise visual synthesis, or (ii) utilize generation as a cognitive scaffold for analytical reasoning. Uni-MMMU incorporates verifiable intermediate reasoning steps, unique ground truths, and a reproducible scoring protocol for both textual and visual outputs. Through extensive evaluation of state-of-the-art unified, generation-only, and understanding-only models, we reveal substantial performance disparities and cross-modal dependencies, offering new insights into when and how these abilities reinforce one another, and establishing a reliable foundation for advancing unified models. |
| title | Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.13759 |