Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Masking
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929520114663424 |
|---|---|
| author | Dong, Zijian Li, Ruilin Wu, Yilei Nguyen, Thuan Tinh Chong, Joanna Su Xian Ji, Fang Tong, Nathanael Ren Jie Chen, Christopher Li Hsian Zhou, Juan Helen |
| author_facet | Dong, Zijian Li, Ruilin Wu, Yilei Nguyen, Thuan Tinh Chong, Joanna Su Xian Ji, Fang Tong, Nathanael Ren Jie Chen, Christopher Li Hsian Zhou, Juan Helen |
| contents | We introduce Brain-JEPA, a brain dynamics foundation model with the Joint-Embedding Predictive Architecture (JEPA). This pioneering model achieves state-of-the-art performance in demographic prediction, disease diagnosis/prognosis, and trait prediction through fine-tuning. Furthermore, it excels in off-the-shelf evaluations (e.g., linear probing) and demonstrates superior generalizability across different ethnic groups, surpassing the previous large model for brain activity significantly. Brain-JEPA incorporates two innovative techniques: Brain Gradient Positioning and Spatiotemporal Masking. Brain Gradient Positioning introduces a functional coordinate system for brain functional parcellation, enhancing the positional encoding of different Regions of Interest (ROIs). Spatiotemporal Masking, tailored to the unique characteristics of fMRI data, addresses the challenge of heterogeneous time-series patches. These methodologies enhance model performance and advance our understanding of the neural circuits underlying cognition. Overall, Brain-JEPA is paving the way to address pivotal questions of building brain functional coordinate system and masking brain activity at the AI-neuroscience interface, and setting a potentially new paradigm in brain activity analysis through downstream adaptation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_19407 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Masking Dong, Zijian Li, Ruilin Wu, Yilei Nguyen, Thuan Tinh Chong, Joanna Su Xian Ji, Fang Tong, Nathanael Ren Jie Chen, Christopher Li Hsian Zhou, Juan Helen Neurons and Cognition Artificial Intelligence Computer Vision and Pattern Recognition We introduce Brain-JEPA, a brain dynamics foundation model with the Joint-Embedding Predictive Architecture (JEPA). This pioneering model achieves state-of-the-art performance in demographic prediction, disease diagnosis/prognosis, and trait prediction through fine-tuning. Furthermore, it excels in off-the-shelf evaluations (e.g., linear probing) and demonstrates superior generalizability across different ethnic groups, surpassing the previous large model for brain activity significantly. Brain-JEPA incorporates two innovative techniques: Brain Gradient Positioning and Spatiotemporal Masking. Brain Gradient Positioning introduces a functional coordinate system for brain functional parcellation, enhancing the positional encoding of different Regions of Interest (ROIs). Spatiotemporal Masking, tailored to the unique characteristics of fMRI data, addresses the challenge of heterogeneous time-series patches. These methodologies enhance model performance and advance our understanding of the neural circuits underlying cognition. Overall, Brain-JEPA is paving the way to address pivotal questions of building brain functional coordinate system and masking brain activity at the AI-neuroscience interface, and setting a potentially new paradigm in brain activity analysis through downstream adaptation. |
| title | Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Masking |
| topic | Neurons and Cognition Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.19407 |