Bridging Perception and Language: A Systematic Benchmark for LVLMs' Understanding of Amodal Completion Reports
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916832308363264 |
|---|---|
| author | Watahiki, Amane Doi, Tomoki Shinozaki, Taiga Nishida, Satoshi Niikawa, Takuya Miyahara, Katsunori Yanaka, Hitomi |
| author_facet | Watahiki, Amane Doi, Tomoki Shinozaki, Taiga Nishida, Satoshi Niikawa, Takuya Miyahara, Katsunori Yanaka, Hitomi |
| contents | One of the main objectives in developing large vision-language models (LVLMs) is to engineer systems that can assist humans with multimodal tasks, including interpreting descriptions of perceptual experiences. A central phenomenon in this context is amodal completion, in which people perceive objects even when parts of those objects are hidden. Although numerous studies have assessed whether computer-vision algorithms can detect or reconstruct occluded regions, the inferential abilities of LVLMs on texts related to amodal completion remain unexplored. To address this gap, we constructed a benchmark grounded in Basic Formal Ontology to achieve a systematic classification of amodal completion. Our results indicate that while many LVLMs achieve human-comparable performance overall, their accuracy diverges for certain types of objects being completed. Notably, in certain categories, some LLaVA-NeXT variants and Claude 3.5 Sonnet exhibit lower accuracy on original images compared to blank stimuli lacking visual content. Intriguingly, this disparity emerges only under Japanese prompting, suggesting a deficiency in Japanese-specific linguistic competence among these models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_05799 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bridging Perception and Language: A Systematic Benchmark for LVLMs' Understanding of Amodal Completion Reports Watahiki, Amane Doi, Tomoki Shinozaki, Taiga Nishida, Satoshi Niikawa, Takuya Miyahara, Katsunori Yanaka, Hitomi Computation and Language One of the main objectives in developing large vision-language models (LVLMs) is to engineer systems that can assist humans with multimodal tasks, including interpreting descriptions of perceptual experiences. A central phenomenon in this context is amodal completion, in which people perceive objects even when parts of those objects are hidden. Although numerous studies have assessed whether computer-vision algorithms can detect or reconstruct occluded regions, the inferential abilities of LVLMs on texts related to amodal completion remain unexplored. To address this gap, we constructed a benchmark grounded in Basic Formal Ontology to achieve a systematic classification of amodal completion. Our results indicate that while many LVLMs achieve human-comparable performance overall, their accuracy diverges for certain types of objects being completed. Notably, in certain categories, some LLaVA-NeXT variants and Claude 3.5 Sonnet exhibit lower accuracy on original images compared to blank stimuli lacking visual content. Intriguingly, this disparity emerges only under Japanese prompting, suggesting a deficiency in Japanese-specific linguistic competence among these models. |
| title | Bridging Perception and Language: A Systematic Benchmark for LVLMs' Understanding of Amodal Completion Reports |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.05799 |