VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909586653446144 |
|---|---|
| author | Xu, Weiye Wang, Jiahao Wang, Weiyun Chen, Zhe Zhou, Wengang Yang, Aijun Lu, Lewei Li, Houqiang Wang, Xiaohua Zhu, Xizhou Wang, Wenhai Dai, Jifeng Zhu, Jinguo |
| author_facet | Xu, Weiye Wang, Jiahao Wang, Weiyun Chen, Zhe Zhou, Wengang Yang, Aijun Lu, Lewei Li, Houqiang Wang, Xiaohua Zhu, Xizhou Wang, Wenhai Dai, Jifeng Zhu, Jinguo |
| contents | Visual reasoning is a core component of human intelligence and a critical capability for advanced multimodal models. Yet current reasoning evaluations of multimodal large language models (MLLMs) often rely on text descriptions and allow language-based reasoning shortcuts, failing to measure genuine vision-centric reasoning. To address this, we introduce VisuLogic: a benchmark of 1,000 human-verified problems across six categories (e.g., quantitative shifts, spatial relations, attribute comparisons). These various types of questions can be evaluated to assess the visual reasoning capabilities of MLLMs from multiple perspectives. We evaluate leading MLLMs on this benchmark and analyze their results to identify common failure modes. Most models score below 30% accuracy-only slightly above the 25% random baseline and far below the 51.4% achieved by humans-revealing significant gaps in visual reasoning. Furthermore, we provide a supplementary training dataset and a reinforcement-learning baseline to support further progress. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_15279 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models Xu, Weiye Wang, Jiahao Wang, Weiyun Chen, Zhe Zhou, Wengang Yang, Aijun Lu, Lewei Li, Houqiang Wang, Xiaohua Zhu, Xizhou Wang, Wenhai Dai, Jifeng Zhu, Jinguo Computer Vision and Pattern Recognition Visual reasoning is a core component of human intelligence and a critical capability for advanced multimodal models. Yet current reasoning evaluations of multimodal large language models (MLLMs) often rely on text descriptions and allow language-based reasoning shortcuts, failing to measure genuine vision-centric reasoning. To address this, we introduce VisuLogic: a benchmark of 1,000 human-verified problems across six categories (e.g., quantitative shifts, spatial relations, attribute comparisons). These various types of questions can be evaluated to assess the visual reasoning capabilities of MLLMs from multiple perspectives. We evaluate leading MLLMs on this benchmark and analyze their results to identify common failure modes. Most models score below 30% accuracy-only slightly above the 25% random baseline and far below the 51.4% achieved by humans-revealing significant gaps in visual reasoning. Furthermore, we provide a supplementary training dataset and a reinforcement-learning baseline to support further progress. |
| title | VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.15279 |