Learning Brain Representation with Hierarchical Visual Embeddings
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912887374610432 |
|---|---|
| author | Zheng, Jiawen Jia, Haonan Li, Ming Zheng, Yuhui Zeng, Yufeng Gao, Yang Liang, Chen |
| author_facet | Zheng, Jiawen Jia, Haonan Li, Ming Zheng, Yuhui Zeng, Yufeng Gao, Yang Liang, Chen |
| contents | Decoding visual representations from brain signals has attracted significant attention in both neuroscience and artificial intelligence. However, the degree to which brain signals truly encode visual information remains unclear. Current visual decoding approaches explore various brain-image alignment strategies, yet most emphasize high-level semantic features while neglecting pixel-level details, thereby limiting our understanding of the human visual system. In this paper, we propose a brain-image alignment strategy that leverages multiple pre-trained visual encoders with distinct inductive biases to capture hierarchical and multi-scale visual representations, while employing a contrastive learning objective to achieve effective alignment between brain signals and visual embeddings. Furthermore, we introduce a Fusion Prior, which learns a stable mapping on large-scale visual data and subsequently matches brain features to this pre-trained prior, thereby enhancing distributional consistency across modalities. Extensive quantitative and qualitative experiments demonstrate that our method achieves a favorable balance between retrieval accuracy and reconstruction fidelity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_07495 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Learning Brain Representation with Hierarchical Visual Embeddings Zheng, Jiawen Jia, Haonan Li, Ming Zheng, Yuhui Zeng, Yufeng Gao, Yang Liang, Chen Computer Vision and Pattern Recognition Multimedia Decoding visual representations from brain signals has attracted significant attention in both neuroscience and artificial intelligence. However, the degree to which brain signals truly encode visual information remains unclear. Current visual decoding approaches explore various brain-image alignment strategies, yet most emphasize high-level semantic features while neglecting pixel-level details, thereby limiting our understanding of the human visual system. In this paper, we propose a brain-image alignment strategy that leverages multiple pre-trained visual encoders with distinct inductive biases to capture hierarchical and multi-scale visual representations, while employing a contrastive learning objective to achieve effective alignment between brain signals and visual embeddings. Furthermore, we introduce a Fusion Prior, which learns a stable mapping on large-scale visual data and subsequently matches brain features to this pre-trained prior, thereby enhancing distributional consistency across modalities. Extensive quantitative and qualitative experiments demonstrate that our method achieves a favorable balance between retrieval accuracy and reconstruction fidelity. |
| title | Learning Brain Representation with Hierarchical Visual Embeddings |
| topic | Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2602.07495 |