Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.17495 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910064509452288 |
|---|---|
| author | Li, Rang Li, Lei Ren, Shuhuai Tian, Hao Gu, Shuhao Li, Shicheng Yue, Zihao Wang, Yudong Ma, Wenhan Yang, Zhe Ma, Jingyuan Sui, Zhifang Luo, Fuli |
| author_facet | Li, Rang Li, Lei Ren, Shuhuai Tian, Hao Gu, Shuhao Li, Shicheng Yue, Zihao Wang, Yudong Ma, Wenhan Yang, Zhe Ma, Jingyuan Sui, Zhifang Luo, Fuli |
| contents | Visual grounding, localizing objects from natural language descriptions, represents a critical bridge between language and vision understanding. While multimodal large language models (MLLMs) achieve impressive scores on existing benchmarks, a fundamental question remains: can MLLMs truly visually ground with human-like sophistication, or are they merely pattern-matching on simplified datasets? Current benchmarks fail to capture real-world complexity where humans effortlessly navigate intricate references and recognize when grounding is impossible. To rigorously assess MLLMs' true capabilities, we introduce GroundingME, a benchmark that systematically challenges models across four critical dimensions: (1) Discriminative: distinguishing highly similar objects, (2) Spatial: understanding complex relational descriptions, (3) Limited: handling occlusions or tiny objects, and (4) Rejection: recognizing ungroundable queries. Through careful curation combining automated generation with human verification, we create 1,005 challenging examples mirroring real-world complexity. Evaluating 25 state-of-the-art MLLMs reveals a profound capability gap: the best model achieves only 45.1% accuracy, while most score 0% on rejection tasks. We explore two strategies for improvements: (1) test-time scaling selects optimal response by thinking trajectory to improve overall performance by up to 4.5%, and (2) data-mixture training boosts rejection accuracy from 0% to 27.9%. GroundingME thus serves as both a diagnostic tool revealing current limitations in MLLMs and a roadmap toward human-level visual grounding. Project page: https://groundingme.github.io |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_17495 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation Li, Rang Li, Lei Ren, Shuhuai Tian, Hao Gu, Shuhao Li, Shicheng Yue, Zihao Wang, Yudong Ma, Wenhan Yang, Zhe Ma, Jingyuan Sui, Zhifang Luo, Fuli Computer Vision and Pattern Recognition Visual grounding, localizing objects from natural language descriptions, represents a critical bridge between language and vision understanding. While multimodal large language models (MLLMs) achieve impressive scores on existing benchmarks, a fundamental question remains: can MLLMs truly visually ground with human-like sophistication, or are they merely pattern-matching on simplified datasets? Current benchmarks fail to capture real-world complexity where humans effortlessly navigate intricate references and recognize when grounding is impossible. To rigorously assess MLLMs' true capabilities, we introduce GroundingME, a benchmark that systematically challenges models across four critical dimensions: (1) Discriminative: distinguishing highly similar objects, (2) Spatial: understanding complex relational descriptions, (3) Limited: handling occlusions or tiny objects, and (4) Rejection: recognizing ungroundable queries. Through careful curation combining automated generation with human verification, we create 1,005 challenging examples mirroring real-world complexity. Evaluating 25 state-of-the-art MLLMs reveals a profound capability gap: the best model achieves only 45.1% accuracy, while most score 0% on rejection tasks. We explore two strategies for improvements: (1) test-time scaling selects optimal response by thinking trajectory to improve overall performance by up to 4.5%, and (2) data-mixture training boosts rejection accuracy from 0% to 27.9%. GroundingME thus serves as both a diagnostic tool revealing current limitations in MLLMs and a roadmap toward human-level visual grounding. Project page: https://groundingme.github.io |
| title | GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.17495 |