Seeing the imagined: a latent functional alignment in visual imagery decoding from fMRI data
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910136397725696 |
|---|---|
| author | Spera, Fabrizio Boccato, Tommaso Olak, Michal Cammarota, Sara Ciferri, Matteo Tronti, Michelangelo Toschi, Nicola Ferrante, Matteo |
| author_facet | Spera, Fabrizio Boccato, Tommaso Olak, Michal Cammarota, Sara Ciferri, Matteo Tronti, Michelangelo Toschi, Nicola Ferrante, Matteo |
| contents | Recent progress in visual brain decoding from fMRI has been enabled by large-scale datasets such as the Natural Scenes Dataset (NSD) and powerful diffusion-based generative models. While current pipelines are primarily optimized for perception, their performance under mental-imagery remains less well understood. In this work, we study how a state-of-the-art (SOTA) perception decoder (DynaDiff) can be adapted to reconstruct imagined content from the Imagery-NSD benchmark. We propose a latent functional alignment approach that maps imagery-evoked activity into the pretrained model's conditioning space, while keeping the remaining components frozen. To mitigate the limited amount of matched imagery-perception supervision, we further introduce a retrieval-based augmentation strategy that selects semantically related NSD perception trials. Across four subjects, latent functional alignment consistently improves high-level semantic reconstruction metrics relative to the frozen pretrained baseline and a voxel-space ridge alignment baseline, and enables above-chance decoding from multiple cortical regions. These results suggest that semantic structure learned from perception can be leveraged to stabilize and improve visual imagery decoding under out-of-distribution conditions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_15374 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Seeing the imagined: a latent functional alignment in visual imagery decoding from fMRI data Spera, Fabrizio Boccato, Tommaso Olak, Michal Cammarota, Sara Ciferri, Matteo Tronti, Michelangelo Toschi, Nicola Ferrante, Matteo Neurons and Cognition Artificial Intelligence Image and Video Processing Recent progress in visual brain decoding from fMRI has been enabled by large-scale datasets such as the Natural Scenes Dataset (NSD) and powerful diffusion-based generative models. While current pipelines are primarily optimized for perception, their performance under mental-imagery remains less well understood. In this work, we study how a state-of-the-art (SOTA) perception decoder (DynaDiff) can be adapted to reconstruct imagined content from the Imagery-NSD benchmark. We propose a latent functional alignment approach that maps imagery-evoked activity into the pretrained model's conditioning space, while keeping the remaining components frozen. To mitigate the limited amount of matched imagery-perception supervision, we further introduce a retrieval-based augmentation strategy that selects semantically related NSD perception trials. Across four subjects, latent functional alignment consistently improves high-level semantic reconstruction metrics relative to the frozen pretrained baseline and a voxel-space ridge alignment baseline, and enables above-chance decoding from multiple cortical regions. These results suggest that semantic structure learned from perception can be leveraged to stabilize and improve visual imagery decoding under out-of-distribution conditions. |
| title | Seeing the imagined: a latent functional alignment in visual imagery decoding from fMRI data |
| topic | Neurons and Cognition Artificial Intelligence Image and Video Processing |
| url | https://arxiv.org/abs/2604.15374 |