From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917457149558784 |
|---|---|
| author | Zhang, Le Yang, Jihan Krishnan, Soundarya Majmudar, Jimit Ge, Xiou Puri, Prasoon Saraf, Prathamesh Nandkishor Bhargava, Shruti Piraviperumal, Dhivya Ling, Yinan Pan, Cindy Yu, Hong Agrawal, Aishwarya Tseng, Bo-Hsiang |
| author_facet | Zhang, Le Yang, Jihan Krishnan, Soundarya Majmudar, Jimit Ge, Xiou Puri, Prasoon Saraf, Prathamesh Nandkishor Bhargava, Shruti Piraviperumal, Dhivya Ling, Yinan Pan, Cindy Yu, Hong Agrawal, Aishwarya Tseng, Bo-Hsiang |
| contents | Human-level agentic intelligence extends beyond low-level geometric perception, evolving from recognizing where things are to understanding what they are for. While existing benchmarks effectively evaluate the geometric perception capabilities of multimodal large language models (MLLMs), they fall short of probing the higher-order cognitive abilities required for grounded intelligence. To address this gap, we introduce the Spatial-Functional Intelligence Benchmark (SFI-Bench), a video-based benchmark with over 1,500 expert-annotated questions derived from diverse egocentric indoor video scans. SFI-Bench systematically evaluates two complementary dimensions of advanced reasoning: (1) Structured Spatial Reasoning, which requires understanding complex layouts and forming coherent spatial representations, and (2) Functional Reasoning, which involves inferring object affordances and their context-dependent utility. The benchmark includes tasks such as conditional counting, multi-hop relational reasoning, functional pairing, and knowledge-grounded troubleshooting, directly challenging models to integrate perception, memory, and inference. Our experiments reveal that current MLLMs consistently struggle to combine spatial memory with functional reasoning and external knowledge, highlighting a critical bottleneck in achieving grounded intelligence. SFI-Bench therefore provides a diagnostic tool for measuring progress toward more cognitively capable and truly grounded multimodal agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_02130 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs Zhang, Le Yang, Jihan Krishnan, Soundarya Majmudar, Jimit Ge, Xiou Puri, Prasoon Saraf, Prathamesh Nandkishor Bhargava, Shruti Piraviperumal, Dhivya Ling, Yinan Pan, Cindy Yu, Hong Agrawal, Aishwarya Tseng, Bo-Hsiang Computer Vision and Pattern Recognition Human-level agentic intelligence extends beyond low-level geometric perception, evolving from recognizing where things are to understanding what they are for. While existing benchmarks effectively evaluate the geometric perception capabilities of multimodal large language models (MLLMs), they fall short of probing the higher-order cognitive abilities required for grounded intelligence. To address this gap, we introduce the Spatial-Functional Intelligence Benchmark (SFI-Bench), a video-based benchmark with over 1,500 expert-annotated questions derived from diverse egocentric indoor video scans. SFI-Bench systematically evaluates two complementary dimensions of advanced reasoning: (1) Structured Spatial Reasoning, which requires understanding complex layouts and forming coherent spatial representations, and (2) Functional Reasoning, which involves inferring object affordances and their context-dependent utility. The benchmark includes tasks such as conditional counting, multi-hop relational reasoning, functional pairing, and knowledge-grounded troubleshooting, directly challenging models to integrate perception, memory, and inference. Our experiments reveal that current MLLMs consistently struggle to combine spatial memory with functional reasoning and external knowledge, highlighting a critical bottleneck in achieving grounded intelligence. SFI-Bench therefore provides a diagnostic tool for measuring progress toward more cognitively capable and truly grounded multimodal agents. |
| title | From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.02130 |