Human-Aligned Image Models Improve Visual Decoding from the Brain
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912421468176384 |
|---|---|
| author | Rajabi, Nona Ribeiro, Antônio H. Vasco, Miguel Taleb, Farzaneh Björkman, Mårten Kragic, Danica |
| author_facet | Rajabi, Nona Ribeiro, Antônio H. Vasco, Miguel Taleb, Farzaneh Björkman, Mårten Kragic, Danica |
| contents | Decoding visual images from brain activity has significant potential for advancing brain-computer interaction and enhancing the understanding of human perception. Recent approaches align the representation spaces of images and brain activity to enable visual decoding. In this paper, we introduce the use of human-aligned image encoders to map brain signals to images. We hypothesize that these models more effectively capture perceptual attributes associated with the rapid visual stimuli presentations commonly used in visual brain data recording experiments. Our empirical results support this hypothesis, demonstrating that this simple modification improves image retrieval accuracy by up to 21% compared to state-of-the-art methods. Comprehensive experiments confirm consistent performance improvements across diverse EEG architectures, image encoders, alignment methods, participants, and brain imaging modalities |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_03081 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Human-Aligned Image Models Improve Visual Decoding from the Brain Rajabi, Nona Ribeiro, Antônio H. Vasco, Miguel Taleb, Farzaneh Björkman, Mårten Kragic, Danica Computer Vision and Pattern Recognition Machine Learning Decoding visual images from brain activity has significant potential for advancing brain-computer interaction and enhancing the understanding of human perception. Recent approaches align the representation spaces of images and brain activity to enable visual decoding. In this paper, we introduce the use of human-aligned image encoders to map brain signals to images. We hypothesize that these models more effectively capture perceptual attributes associated with the rapid visual stimuli presentations commonly used in visual brain data recording experiments. Our empirical results support this hypothesis, demonstrating that this simple modification improves image retrieval accuracy by up to 21% compared to state-of-the-art methods. Comprehensive experiments confirm consistent performance improvements across diverse EEG architectures, image encoders, alignment methods, participants, and brain imaging modalities |
| title | Human-Aligned Image Models Improve Visual Decoding from the Brain |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2502.03081 |