Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.08303 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917324710215680 |
|---|---|
| author | Xiao, Xin Lei, Yang Zeng, Haoyang Sun, Xiao Jiang, Xinyi Tian, Yu Wu, Hao Wei, Kaiwen Zhong, Jiang |
| author_facet | Xiao, Xin Lei, Yang Zeng, Haoyang Sun, Xiao Jiang, Xinyi Tian, Yu Wu, Hao Wei, Kaiwen Zhong, Jiang |
| contents | Large Vision Language Models (LVLMs) exhibit strong visual understanding and reasoning abilities. However, whether their internal representations reflect human visual cognition is still under-explored. In this paper, we address this by quantifying LVLM-brain alignment using image-evoked Electroencephalogram (EEG) signals, analyzing the effects of model architecture, scale, and image type. Specifically, by using ridge regression and representational similarity analysis, we compare visual representations from 32 open-source LVLMs with corresponding EEG responses. We observe a structured LVLM-brain correspondence: First, intermediate layers (8-16) show peak alignment with EEG activity in the 100-300 ms window, consistent with hierarchical human visual processing. Secondly, multimodal architectural design contributes 3.4 more to brain alignment than parameter scaling, and models with stronger downstream visual performance exhibit higher EEG similarity. Thirdly, spatiotemporal patterns further align with known cortical visual pathways. These results demonstrate that LVLMs learn human-aligned visual representations and establish neural alignment as a biologically grounded benchmark for evaluating and improving LVLMs. In addition, those results could provide insights that may inform the development of neuro-inspired applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_08303 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Do Models See in Line with Human Vision? Probing the Correspondence Between LVLM Representations and EEG Signals Xiao, Xin Lei, Yang Zeng, Haoyang Sun, Xiao Jiang, Xinyi Tian, Yu Wu, Hao Wei, Kaiwen Zhong, Jiang Human-Computer Interaction Large Vision Language Models (LVLMs) exhibit strong visual understanding and reasoning abilities. However, whether their internal representations reflect human visual cognition is still under-explored. In this paper, we address this by quantifying LVLM-brain alignment using image-evoked Electroencephalogram (EEG) signals, analyzing the effects of model architecture, scale, and image type. Specifically, by using ridge regression and representational similarity analysis, we compare visual representations from 32 open-source LVLMs with corresponding EEG responses. We observe a structured LVLM-brain correspondence: First, intermediate layers (8-16) show peak alignment with EEG activity in the 100-300 ms window, consistent with hierarchical human visual processing. Secondly, multimodal architectural design contributes 3.4 more to brain alignment than parameter scaling, and models with stronger downstream visual performance exhibit higher EEG similarity. Thirdly, spatiotemporal patterns further align with known cortical visual pathways. These results demonstrate that LVLMs learn human-aligned visual representations and establish neural alignment as a biologically grounded benchmark for evaluating and improving LVLMs. In addition, those results could provide insights that may inform the development of neuro-inspired applications. |
| title | Do Models See in Line with Human Vision? Probing the Correspondence Between LVLM Representations and EEG Signals |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2603.08303 |