Scaling Vision Transformers for Functional MRI with Flat Maps
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917455678406656 |
|---|---|
| author | Lane, Connor Tripathy, Mihir Murali, Leema Krishna Grandhi, Ratna Sagari Yang, Shamus Sim Zi Gijsen, Sam Das, Debojyoti Ram, Manish Singh, Utkarsh Kumar Villanueva, Cesar Kadir Torrico Wei, Yuxiang Beddow, Will Cortés, Gianfranco Cho, Suin Kaplan, Daniel Z. Warner, Benjamin Abraham, Tanishq Mathew Scotti, Paul S. |
| author_facet | Lane, Connor Tripathy, Mihir Murali, Leema Krishna Grandhi, Ratna Sagari Yang, Shamus Sim Zi Gijsen, Sam Das, Debojyoti Ram, Manish Singh, Utkarsh Kumar Villanueva, Cesar Kadir Torrico Wei, Yuxiang Beddow, Will Cortés, Gianfranco Cho, Suin Kaplan, Daniel Z. Warner, Benjamin Abraham, Tanishq Mathew Scotti, Paul S. |
| contents | We study the problem of training self-supervised foundation models for functional MRI. Our main contributions are: (1) we introduce a new model family (CortexMAE) trained using the masked autoencoder framework on 2.1K hours of open fMRI data, and (2) we release the first open evaluation suite (Brainmarks) for fMRI foundation models. Our core innovation is simple: we adapt the Vision Transformer to fMRI by first converting each 3D fMRI volume to a 2D map using a cortical flat map projection. We directly compare flat maps to both parcellation and volume-based representations. While each has its advantages, flat maps generally perform best. We perform the first systematic scaling analysis for fMRI and observe strict power law scaling, albeit with limits. Finally, we use Brainmarks to do controlled benchmark comparisons. On subject-level trait prediction, we report a challenging null result: no single model achieves clear state-of-the-art performance. Moreover, all models struggle to outperform a simple functional connectivity baseline. On cognitive state decoding, we observe more robust performance, and in this setting our CortexMAE family outperforms prior models by a large margin. Code, models, and datasets are available at https://github.com/MedARC-AI/CortexMAE and https://github.com/MedARC-AI/Brainmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_13768 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scaling Vision Transformers for Functional MRI with Flat Maps Lane, Connor Tripathy, Mihir Murali, Leema Krishna Grandhi, Ratna Sagari Yang, Shamus Sim Zi Gijsen, Sam Das, Debojyoti Ram, Manish Singh, Utkarsh Kumar Villanueva, Cesar Kadir Torrico Wei, Yuxiang Beddow, Will Cortés, Gianfranco Cho, Suin Kaplan, Daniel Z. Warner, Benjamin Abraham, Tanishq Mathew Scotti, Paul S. Computer Vision and Pattern Recognition Artificial Intelligence Neurons and Cognition We study the problem of training self-supervised foundation models for functional MRI. Our main contributions are: (1) we introduce a new model family (CortexMAE) trained using the masked autoencoder framework on 2.1K hours of open fMRI data, and (2) we release the first open evaluation suite (Brainmarks) for fMRI foundation models. Our core innovation is simple: we adapt the Vision Transformer to fMRI by first converting each 3D fMRI volume to a 2D map using a cortical flat map projection. We directly compare flat maps to both parcellation and volume-based representations. While each has its advantages, flat maps generally perform best. We perform the first systematic scaling analysis for fMRI and observe strict power law scaling, albeit with limits. Finally, we use Brainmarks to do controlled benchmark comparisons. On subject-level trait prediction, we report a challenging null result: no single model achieves clear state-of-the-art performance. Moreover, all models struggle to outperform a simple functional connectivity baseline. On cognitive state decoding, we observe more robust performance, and in this setting our CortexMAE family outperforms prior models by a large margin. Code, models, and datasets are available at https://github.com/MedARC-AI/CortexMAE and https://github.com/MedARC-AI/Brainmarks. |
| title | Scaling Vision Transformers for Functional MRI with Flat Maps |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Neurons and Cognition |
| url | https://arxiv.org/abs/2510.13768 |