UniM: A Unified Any-to-Any Interleaved Multimodal Benchmark
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911488589955072 |
|---|---|
| author | Li, Yanlin Guo, Minghui Zhang, Kaiwen Zhang, Shize Zhao, Yiran Li, Haodong Zhou, Congyue Zheng, Weijie Yan, Yushen Wu, Shengqiong Ji, Wei Cui, Lei Wei, Furu Fei, Hao Lee, Mong-Li Hsu, Wynne |
| author_facet | Li, Yanlin Guo, Minghui Zhang, Kaiwen Zhang, Shize Zhao, Yiran Li, Haodong Zhou, Congyue Zheng, Weijie Yan, Yushen Wu, Shengqiong Ji, Wei Cui, Lei Wei, Furu Fei, Hao Lee, Mong-Li Hsu, Wynne |
| contents | In real-world multimodal applications, systems usually need to comprehend arbitrarily combined and interleaved multimodal inputs from users, while also generating outputs in any interleaved multimedia form. This capability defines the goal of any-to-any interleaved multimodal learning under a unified paradigm of understanding and generation, posing new challenges and opportunities for advancing Multimodal Large Language Models (MLLMs). To foster and benchmark this capability, this paper introduces the UniM benchmark, the first Unified Any-to-Any Interleaved Multimodal dataset. UniM contains 31K high-quality instances across 30 domains and 7 representative modalities: text, image, audio, video, document, code, and 3D, each requiring multiple intertwined reasoning and generation capabilities. We further introduce the UniM Evaluation Suite, which assesses models along three dimensions: Semantic Correctness & Generation Quality, Response Structure Integrity, and Interleaved Coherence. In addition, we propose UniMA, an agentic baseline model equipped with traceable reasoning for structured interleaved generation. Comprehensive experiments demonstrate the difficulty of UniM and highlight key challenges and directions for advancing unified any-to-any multimodal intelligence. The project page is https://any2any-mllm.github.io/unim. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_05075 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | UniM: A Unified Any-to-Any Interleaved Multimodal Benchmark Li, Yanlin Guo, Minghui Zhang, Kaiwen Zhang, Shize Zhao, Yiran Li, Haodong Zhou, Congyue Zheng, Weijie Yan, Yushen Wu, Shengqiong Ji, Wei Cui, Lei Wei, Furu Fei, Hao Lee, Mong-Li Hsu, Wynne Computer Vision and Pattern Recognition In real-world multimodal applications, systems usually need to comprehend arbitrarily combined and interleaved multimodal inputs from users, while also generating outputs in any interleaved multimedia form. This capability defines the goal of any-to-any interleaved multimodal learning under a unified paradigm of understanding and generation, posing new challenges and opportunities for advancing Multimodal Large Language Models (MLLMs). To foster and benchmark this capability, this paper introduces the UniM benchmark, the first Unified Any-to-Any Interleaved Multimodal dataset. UniM contains 31K high-quality instances across 30 domains and 7 representative modalities: text, image, audio, video, document, code, and 3D, each requiring multiple intertwined reasoning and generation capabilities. We further introduce the UniM Evaluation Suite, which assesses models along three dimensions: Semantic Correctness & Generation Quality, Response Structure Integrity, and Interleaved Coherence. In addition, we propose UniMA, an agentic baseline model equipped with traceable reasoning for structured interleaved generation. Comprehensive experiments demonstrate the difficulty of UniM and highlight key challenges and directions for advancing unified any-to-any multimodal intelligence. The project page is https://any2any-mllm.github.io/unim. |
| title | UniM: A Unified Any-to-Any Interleaved Multimodal Benchmark |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.05075 |