Mapping fMRI Signal and Image Stimuli in an Artificial Neural Network Latent Space: Bringing Artificial and Natural Minds Together
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910896113057792 |
|---|---|
| author | Dalbagno, Cesare Maria Jardim, Manuel de Castro Ribeiro Angheluţă, Mihnea |
| author_facet | Dalbagno, Cesare Maria Jardim, Manuel de Castro Ribeiro Angheluţă, Mihnea |
| contents | The goal of this study is to investigate whether latent space representations of visual stimuli and fMRI data share common information. Decoding and reconstructing stimuli from fMRI data remains a challenge in AI and neuroscience, with significant implications for understanding neural representations and improving the interpretability of Artificial Neural Networks (ANNs). In this preliminary study, we investigate the feasibility of such reconstruction by examining the similarity between the latent spaces of one autoencoder (AE) and one vision transformer (ViT) trained on fMRI and image data, respectively. Using representational similarity analysis (RSA), we found that the latent spaces of the two domains appear different. However, these initial findings are inconclusive, and further research is needed to explore this relationship more thoroughly. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_19923 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mapping fMRI Signal and Image Stimuli in an Artificial Neural Network Latent Space: Bringing Artificial and Natural Minds Together Dalbagno, Cesare Maria Jardim, Manuel de Castro Ribeiro Angheluţă, Mihnea Neurons and Cognition Computer Vision and Pattern Recognition 92B20, 68T07, 62P10, 94A12 I.2.6; I.5.1; J.3 The goal of this study is to investigate whether latent space representations of visual stimuli and fMRI data share common information. Decoding and reconstructing stimuli from fMRI data remains a challenge in AI and neuroscience, with significant implications for understanding neural representations and improving the interpretability of Artificial Neural Networks (ANNs). In this preliminary study, we investigate the feasibility of such reconstruction by examining the similarity between the latent spaces of one autoencoder (AE) and one vision transformer (ViT) trained on fMRI and image data, respectively. Using representational similarity analysis (RSA), we found that the latent spaces of the two domains appear different. However, these initial findings are inconclusive, and further research is needed to explore this relationship more thoroughly. |
| title | Mapping fMRI Signal and Image Stimuli in an Artificial Neural Network Latent Space: Bringing Artificial and Natural Minds Together |
| topic | Neurons and Cognition Computer Vision and Pattern Recognition 92B20, 68T07, 62P10, 94A12 I.2.6; I.5.1; J.3 |
| url | https://arxiv.org/abs/2503.19923 |