Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.20426 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909921949253632 |
|---|---|
| author | Tang, Yolo Y. Liu, Pinxin Tan, Zhangyun Feng, Mingqian Mao, Rui Huang, Chao Bi, Jing Xiao, Yunzhong Liang, Susan Hua, Hang Vosoughi, Ali Song, Luchuan Zhang, Zeliang Xu, Chenliang |
| author_facet | Tang, Yolo Y. Liu, Pinxin Tan, Zhangyun Feng, Mingqian Mao, Rui Huang, Chao Bi, Jing Xiao, Yunzhong Liang, Susan Hua, Hang Vosoughi, Ali Song, Luchuan Zhang, Zeliang Xu, Chenliang |
| contents | Understanding perspective is fundamental to human visual perception, yet the extent to which multimodal large language models (MLLMs) internalize perspective geometry remains unclear. We introduce MMPerspective, the first benchmark specifically designed to systematically evaluate MLLMs' understanding of perspective through 10 carefully crafted tasks across three complementary dimensions: Perspective Perception, Reasoning, and Robustness. Our benchmark comprises 2,711 real-world and synthetic image instances with 5,083 question-answer pairs that probe key capabilities, such as vanishing point perception and counting, perspective type reasoning, line relationship understanding in 3D space, invariance to perspective-preserving transformations, etc. Through a comprehensive evaluation of 43 state-of-the-art MLLMs, we uncover significant limitations: while models demonstrate competence on surface-level perceptual tasks, they struggle with compositional reasoning and maintaining spatial consistency under perturbations. Our analysis further reveals intriguing patterns between model architecture, scale, and perspective capabilities, highlighting both robustness bottlenecks and the benefits of chain-of-thought prompting. MMPerspective establishes a valuable testbed for diagnosing and advancing spatial understanding in vision-language systems. Resources available at: https://yunlong10.github.io/MMPerspective/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_20426 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MMPerspective: Do MLLMs Understand Perspective? A Comprehensive Benchmark for Perspective Perception, Reasoning, and Robustness Tang, Yolo Y. Liu, Pinxin Tan, Zhangyun Feng, Mingqian Mao, Rui Huang, Chao Bi, Jing Xiao, Yunzhong Liang, Susan Hua, Hang Vosoughi, Ali Song, Luchuan Zhang, Zeliang Xu, Chenliang Computer Vision and Pattern Recognition Understanding perspective is fundamental to human visual perception, yet the extent to which multimodal large language models (MLLMs) internalize perspective geometry remains unclear. We introduce MMPerspective, the first benchmark specifically designed to systematically evaluate MLLMs' understanding of perspective through 10 carefully crafted tasks across three complementary dimensions: Perspective Perception, Reasoning, and Robustness. Our benchmark comprises 2,711 real-world and synthetic image instances with 5,083 question-answer pairs that probe key capabilities, such as vanishing point perception and counting, perspective type reasoning, line relationship understanding in 3D space, invariance to perspective-preserving transformations, etc. Through a comprehensive evaluation of 43 state-of-the-art MLLMs, we uncover significant limitations: while models demonstrate competence on surface-level perceptual tasks, they struggle with compositional reasoning and maintaining spatial consistency under perturbations. Our analysis further reveals intriguing patterns between model architecture, scale, and perspective capabilities, highlighting both robustness bottlenecks and the benefits of chain-of-thought prompting. MMPerspective establishes a valuable testbed for diagnosing and advancing spatial understanding in vision-language systems. Resources available at: https://yunlong10.github.io/MMPerspective/ |
| title | MMPerspective: Do MLLMs Understand Perspective? A Comprehensive Benchmark for Perspective Perception, Reasoning, and Robustness |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.20426 |