Hidden in Plain Sight: Evaluating Abstract Shape Recognition in Vision-Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909383028375552 |
|---|---|
| author | Hemmat, Arshia Davies, Adam Lamb, Tom A. Yuan, Jianhao Torr, Philip Khakzar, Ashkan Pinto, Francesco |
| author_facet | Hemmat, Arshia Davies, Adam Lamb, Tom A. Yuan, Jianhao Torr, Philip Khakzar, Ashkan Pinto, Francesco |
| contents | Despite the importance of shape perception in human vision, early neural image classifiers relied less on shape information for object recognition than other (often spurious) features. While recent research suggests that current large Vision-Language Models (VLMs) exhibit more reliance on shape, we find them to still be seriously limited in this regard. To quantify such limitations, we introduce IllusionBench, a dataset that challenges current cutting-edge VLMs to decipher shape information when the shape is represented by an arrangement of visual elements in a scene. Our extensive evaluations reveal that, while these shapes are easily detectable by human annotators, current VLMs struggle to recognize them, indicating important avenues for future work in developing more robust visual perception systems. The full dataset and codebase are available at: \url{https://arshiahemmat.github.io/illusionbench/} |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_06287 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hidden in Plain Sight: Evaluating Abstract Shape Recognition in Vision-Language Models Hemmat, Arshia Davies, Adam Lamb, Tom A. Yuan, Jianhao Torr, Philip Khakzar, Ashkan Pinto, Francesco Computer Vision and Pattern Recognition Despite the importance of shape perception in human vision, early neural image classifiers relied less on shape information for object recognition than other (often spurious) features. While recent research suggests that current large Vision-Language Models (VLMs) exhibit more reliance on shape, we find them to still be seriously limited in this regard. To quantify such limitations, we introduce IllusionBench, a dataset that challenges current cutting-edge VLMs to decipher shape information when the shape is represented by an arrangement of visual elements in a scene. Our extensive evaluations reveal that, while these shapes are easily detectable by human annotators, current VLMs struggle to recognize them, indicating important avenues for future work in developing more robust visual perception systems. The full dataset and codebase are available at: \url{https://arshiahemmat.github.io/illusionbench/} |
| title | Hidden in Plain Sight: Evaluating Abstract Shape Recognition in Vision-Language Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.06287 |