RefBench-PRO: Perceptual and Reasoning Oriented Benchmark for Referring Expression Comprehension
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908708068392960 |
|---|---|
| author | Gao, Tianyi Li, Hao Fang, Han Wei, Xin Dong, Xiaodong Sun, Hongbo Yuan, Ye He, Zhongjiang Xu, Jinglin Xin, Jingmin Sun, Hao |
| author_facet | Gao, Tianyi Li, Hao Fang, Han Wei, Xin Dong, Xiaodong Sun, Hongbo Yuan, Ye He, Zhongjiang Xu, Jinglin Xin, Jingmin Sun, Hao |
| contents | Referring Expression Comprehension (REC) is a vision-language task that localizes a specific image region based on a textual description. Existing REC benchmarks primarily evaluate perceptual capabilities and lack interpretable scoring mechanisms, which cannot reveal the grounding capability of Multi-modal Large Language Model (MLLM) across different cognitive abilities. To address this limitation, we introduce RefBench-PRO, a comprehensive REC benchmark, which decomposes referring expressions into two core dimensions, i.e., perception and reasoning, and further subdivides them into six progressively challenging tasks, such as attribute, position, interaction, commonsense, relation and reject. We also develop a fully automated data-generation pipeline that produces diverse referring expressions across these six sub-dimensions. Furthermore, We propose Ref-R1, an RL-based learning scheme, which incorporates Dynamic IoU-based GRPO to improve localization accuracy under increasingly complex reasoning conditions, establishing a stronger baseline for REC. Extensive experiments demonstrate that our RefBench-PRO enables interpretable evaluation of MLLM on referring expression comprehension, presenting greater challenges in both perception and reasoning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_06276 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RefBench-PRO: Perceptual and Reasoning Oriented Benchmark for Referring Expression Comprehension Gao, Tianyi Li, Hao Fang, Han Wei, Xin Dong, Xiaodong Sun, Hongbo Yuan, Ye He, Zhongjiang Xu, Jinglin Xin, Jingmin Sun, Hao Computer Vision and Pattern Recognition Artificial Intelligence Referring Expression Comprehension (REC) is a vision-language task that localizes a specific image region based on a textual description. Existing REC benchmarks primarily evaluate perceptual capabilities and lack interpretable scoring mechanisms, which cannot reveal the grounding capability of Multi-modal Large Language Model (MLLM) across different cognitive abilities. To address this limitation, we introduce RefBench-PRO, a comprehensive REC benchmark, which decomposes referring expressions into two core dimensions, i.e., perception and reasoning, and further subdivides them into six progressively challenging tasks, such as attribute, position, interaction, commonsense, relation and reject. We also develop a fully automated data-generation pipeline that produces diverse referring expressions across these six sub-dimensions. Furthermore, We propose Ref-R1, an RL-based learning scheme, which incorporates Dynamic IoU-based GRPO to improve localization accuracy under increasingly complex reasoning conditions, establishing a stronger baseline for REC. Extensive experiments demonstrate that our RefBench-PRO enables interpretable evaluation of MLLM on referring expression comprehension, presenting greater challenges in both perception and reasoning. |
| title | RefBench-PRO: Perceptual and Reasoning Oriented Benchmark for Referring Expression Comprehension |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2512.06276 |