Interpretable EEG-to-Image Generation with Semantic Prompts
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913934855897088 |
|---|---|
| author | Rezvani, Arshak Akbari, Ali Arani, Kosar Sanjar Mirian, Maryam Arasteh, Emad McKeown, Martin J. |
| author_facet | Rezvani, Arshak Akbari, Ali Arani, Kosar Sanjar Mirian, Maryam Arasteh, Emad McKeown, Martin J. |
| contents | Decoding visual experience from brain signals offers exciting possibilities for neuroscience and interpretable AI. While EEG is accessible and temporally precise, its limitations in spatial detail hinder image reconstruction. Our model bypasses direct EEG-to-image generation by aligning EEG signals with multilevel semantic captions -- ranging from object-level to abstract themes -- generated by a large language model. A transformer-based EEG encoder maps brain activity to these captions through contrastive learning. During inference, caption embeddings retrieved via projection heads condition a pretrained latent diffusion model for image generation. This text-mediated framework yields state-of-the-art visual decoding on the EEGCVPR dataset, with interpretable alignment to known neurocognitive pathways. Dominant EEG-caption associations reflected the importance of different semantic levels extracted from perceived images. Saliency maps and t-SNE projections reveal semantic topography across the scalp. Our model demonstrates how structured semantic mediation enables cognitively aligned visual decoding from EEG. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_07157 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Interpretable EEG-to-Image Generation with Semantic Prompts Rezvani, Arshak Akbari, Ali Arani, Kosar Sanjar Mirian, Maryam Arasteh, Emad McKeown, Martin J. Computer Vision and Pattern Recognition Machine Learning Signal Processing Decoding visual experience from brain signals offers exciting possibilities for neuroscience and interpretable AI. While EEG is accessible and temporally precise, its limitations in spatial detail hinder image reconstruction. Our model bypasses direct EEG-to-image generation by aligning EEG signals with multilevel semantic captions -- ranging from object-level to abstract themes -- generated by a large language model. A transformer-based EEG encoder maps brain activity to these captions through contrastive learning. During inference, caption embeddings retrieved via projection heads condition a pretrained latent diffusion model for image generation. This text-mediated framework yields state-of-the-art visual decoding on the EEGCVPR dataset, with interpretable alignment to known neurocognitive pathways. Dominant EEG-caption associations reflected the importance of different semantic levels extracted from perceived images. Saliency maps and t-SNE projections reveal semantic topography across the scalp. Our model demonstrates how structured semantic mediation enables cognitively aligned visual decoding from EEG. |
| title | Interpretable EEG-to-Image Generation with Semantic Prompts |
| topic | Computer Vision and Pattern Recognition Machine Learning Signal Processing |
| url | https://arxiv.org/abs/2507.07157 |