GePBench: Evaluating Fundamental Geometric Perception for Multimodal Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917924177969152 |
|---|---|
| author | Xing, Shangyu Xiang, Changhao Han, Yuteng Yue, Yifan Wu, Zhen Liu, Xinyu Wu, Zhangtai Zhao, Fei Dai, Xinyu |
| author_facet | Xing, Shangyu Xiang, Changhao Han, Yuteng Yue, Yifan Wu, Zhen Liu, Xinyu Wu, Zhangtai Zhao, Fei Dai, Xinyu |
| contents | Multimodal large language models (MLLMs) have made significant progress in integrating visual and linguistic understanding. Existing benchmarks typically focus on high-level semantic capabilities, such as scene understanding and visual reasoning, but often overlook a crucial, foundational ability: geometric perception. Geometric perception involves understanding geometric shapes, structures, and spatial relationships, which are essential for supporting higher-level semantic tasks. Despite its importance, this capability remains underexplored in current MLLM research. To address this gap, we introduce GePBench, a novel benchmark designed to assess the geometric perception abilities of MLLMs. Our extensive evaluations reveal that current state-of-the-art MLLMs exhibit significant deficiencies in geometric perception tasks. Furthermore, we show that models trained with GePBench data demonstrate substantial improvements on a wide range of benchmark tasks, highlighting the critical role of geometric perception in enabling advanced multimodal applications. Our code and datasets will be publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_21036 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GePBench: Evaluating Fundamental Geometric Perception for Multimodal Large Language Models Xing, Shangyu Xiang, Changhao Han, Yuteng Yue, Yifan Wu, Zhen Liu, Xinyu Wu, Zhangtai Zhao, Fei Dai, Xinyu Computation and Language Multimodal large language models (MLLMs) have made significant progress in integrating visual and linguistic understanding. Existing benchmarks typically focus on high-level semantic capabilities, such as scene understanding and visual reasoning, but often overlook a crucial, foundational ability: geometric perception. Geometric perception involves understanding geometric shapes, structures, and spatial relationships, which are essential for supporting higher-level semantic tasks. Despite its importance, this capability remains underexplored in current MLLM research. To address this gap, we introduce GePBench, a novel benchmark designed to assess the geometric perception abilities of MLLMs. Our extensive evaluations reveal that current state-of-the-art MLLMs exhibit significant deficiencies in geometric perception tasks. Furthermore, we show that models trained with GePBench data demonstrate substantial improvements on a wide range of benchmark tasks, highlighting the critical role of geometric perception in enabling advanced multimodal applications. Our code and datasets will be publicly available. |
| title | GePBench: Evaluating Fundamental Geometric Perception for Multimodal Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.21036 |