VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917168957882368 |
|---|---|
| author | Törtei, Brigitta Malagurski Dahou, Yasser Huynh, Ngoc Dung Para, Wamiq Reyaz Khac, Phúc H. Lê Singh, Ankit Chaybouti, Sofian Narayan, Sanath |
| author_facet | Törtei, Brigitta Malagurski Dahou, Yasser Huynh, Ngoc Dung Para, Wamiq Reyaz Khac, Phúc H. Lê Singh, Ankit Chaybouti, Sofian Narayan, Sanath |
| contents | Vision-Language Models (VLMs) have achieved remarkable progress across tasks such as visual question answering and image captioning. Yet, the extent to which these models perform visual reasoning as opposed to relying on linguistic priors remains unclear. To address this, we introduce VisRes Bench, a benchmark designed to study visual reasoning in naturalistic settings without contextual language supervision. Analyzing model behavior across three levels of complexity, we uncover clear limitations in perceptual and relational visual reasoning capacities. VisRes isolates distinct reasoning abilities across its levels. Level 1 probes perceptual completion and global image matching under perturbations such as blur, texture changes, occlusion, and rotation; Level 2 tests rule-based inference over a single attribute (e.g., color, count, orientation); and Level 3 targets compositional reasoning that requires integrating multiple visual attributes. Across more than 19,000 controlled task images, we find that state-of-the-art VLMs perform near random under subtle perceptual perturbations, revealing limited abstraction beyond pattern recognition. We conclude by discussing how VisRes provides a unified framework for advancing abstract visual reasoning in multimodal research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_21194 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs Törtei, Brigitta Malagurski Dahou, Yasser Huynh, Ngoc Dung Para, Wamiq Reyaz Khac, Phúc H. Lê Singh, Ankit Chaybouti, Sofian Narayan, Sanath Computer Vision and Pattern Recognition Vision-Language Models (VLMs) have achieved remarkable progress across tasks such as visual question answering and image captioning. Yet, the extent to which these models perform visual reasoning as opposed to relying on linguistic priors remains unclear. To address this, we introduce VisRes Bench, a benchmark designed to study visual reasoning in naturalistic settings without contextual language supervision. Analyzing model behavior across three levels of complexity, we uncover clear limitations in perceptual and relational visual reasoning capacities. VisRes isolates distinct reasoning abilities across its levels. Level 1 probes perceptual completion and global image matching under perturbations such as blur, texture changes, occlusion, and rotation; Level 2 tests rule-based inference over a single attribute (e.g., color, count, orientation); and Level 3 targets compositional reasoning that requires integrating multiple visual attributes. Across more than 19,000 controlled task images, we find that state-of-the-art VLMs perform near random under subtle perceptual perturbations, revealing limited abstraction beyond pattern recognition. We conclude by discussing how VisRes provides a unified framework for advancing abstract visual reasoning in multimodal research. |
| title | VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.21194 |