Mind-to-Image: Projecting Visual Mental Imagination of the Brain from fMRI
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913366529802240 |
|---|---|
| author | Caselles-Dupré, Hugo Mellerio, Charles Hérent, Paul Lopez-Persem, Alizée Béranger, Benoit Soularue, Mathieu Fautrel, Pierre Vernier, Gauthier Cord, Matthieu |
| author_facet | Caselles-Dupré, Hugo Mellerio, Charles Hérent, Paul Lopez-Persem, Alizée Béranger, Benoit Soularue, Mathieu Fautrel, Pierre Vernier, Gauthier Cord, Matthieu |
| contents | The reconstruction of images observed by subjects from fMRI data collected during visual stimuli has made strong progress in the past decade, thanks to the availability of extensive fMRI datasets and advancements in generative models for image generation. However, the application of visual reconstruction has remained limited. Reconstructing visual imagination presents a greater challenge, with potentially revolutionary applications ranging from aiding individuals with disabilities to verifying witness accounts in court. The primary hurdles in this field are the absence of data collection protocols for visual imagery and the lack of datasets on the subject. Traditionally, fMRI-to-image relies on data collected from subjects exposed to visual stimuli, which poses issues for generating visual imagery based on the difference of brain activity between visual stimulation and visual imagery. For the first time, we have compiled a substantial dataset (around 6h of scans) on visual imagery along with a proposed data collection protocol. We then train a modified version of an fMRI-to-image model and demonstrate the feasibility of reconstructing images from two modes of imagination: from memory and from pure imagination. The resulting pipeline we call Mind-to-Image marks a step towards creating a technology that allow direct reconstruction of visual imagery. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_05468 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mind-to-Image: Projecting Visual Mental Imagination of the Brain from fMRI Caselles-Dupré, Hugo Mellerio, Charles Hérent, Paul Lopez-Persem, Alizée Béranger, Benoit Soularue, Mathieu Fautrel, Pierre Vernier, Gauthier Cord, Matthieu Neurons and Cognition Computer Vision and Pattern Recognition Machine Learning The reconstruction of images observed by subjects from fMRI data collected during visual stimuli has made strong progress in the past decade, thanks to the availability of extensive fMRI datasets and advancements in generative models for image generation. However, the application of visual reconstruction has remained limited. Reconstructing visual imagination presents a greater challenge, with potentially revolutionary applications ranging from aiding individuals with disabilities to verifying witness accounts in court. The primary hurdles in this field are the absence of data collection protocols for visual imagery and the lack of datasets on the subject. Traditionally, fMRI-to-image relies on data collected from subjects exposed to visual stimuli, which poses issues for generating visual imagery based on the difference of brain activity between visual stimulation and visual imagery. For the first time, we have compiled a substantial dataset (around 6h of scans) on visual imagery along with a proposed data collection protocol. We then train a modified version of an fMRI-to-image model and demonstrate the feasibility of reconstructing images from two modes of imagination: from memory and from pure imagination. The resulting pipeline we call Mind-to-Image marks a step towards creating a technology that allow direct reconstruction of visual imagery. |
| title | Mind-to-Image: Projecting Visual Mental Imagination of the Brain from fMRI |
| topic | Neurons and Cognition Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2404.05468 |