Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.18679 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908725882650624 |
|---|---|
| author | Kayser, Maxime Gridnev, Maksim Wang, Wanting Bain, Max Rangnekar, Aneesh Chatterjee, Avijit Petrov, Aleksandr Veeraraghavan, Harini Swinburne, Nathaniel C. |
| author_facet | Kayser, Maxime Gridnev, Maksim Wang, Wanting Bain, Max Rangnekar, Aneesh Chatterjee, Avijit Petrov, Aleksandr Veeraraghavan, Harini Swinburne, Nathaniel C. |
| contents | We present brat (brain report alignment transformer), a multi-view representation learning framework for brain magnetic resonance imaging (MRI) trained on MRIs paired with clinical reports. Brain MRIs present unique challenges due to the presence of numerous, highly varied, and often subtle abnormalities that are localized to a few slices within a 3D volume. To address these challenges, we introduce a brain MRI dataset $10\times$ larger than existing ones, containing approximately 80,000 3D scans with corresponding radiology reports, and propose a multi-view pre-training approach inspired by advances in document retrieval. We develop an implicit query-feature matching mechanism and adopt concepts from quality-diversity to obtain multi-view embeddings of MRIs that are aligned with the clinical features given by report sentences. We evaluate our approach across multiple vision-language and vision tasks, demonstrating substantial performance improvements. The brat foundation models are publicly released. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_18679 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | brat: Aligned Multi-View Embeddings for Brain MRI Analysis Kayser, Maxime Gridnev, Maksim Wang, Wanting Bain, Max Rangnekar, Aneesh Chatterjee, Avijit Petrov, Aleksandr Veeraraghavan, Harini Swinburne, Nathaniel C. Computer Vision and Pattern Recognition Computation and Language We present brat (brain report alignment transformer), a multi-view representation learning framework for brain magnetic resonance imaging (MRI) trained on MRIs paired with clinical reports. Brain MRIs present unique challenges due to the presence of numerous, highly varied, and often subtle abnormalities that are localized to a few slices within a 3D volume. To address these challenges, we introduce a brain MRI dataset $10\times$ larger than existing ones, containing approximately 80,000 3D scans with corresponding radiology reports, and propose a multi-view pre-training approach inspired by advances in document retrieval. We develop an implicit query-feature matching mechanism and adopt concepts from quality-diversity to obtain multi-view embeddings of MRIs that are aligned with the clinical features given by report sentences. We evaluate our approach across multiple vision-language and vision tasks, demonstrating substantial performance improvements. The brat foundation models are publicly released. |
| title | brat: Aligned Multi-View Embeddings for Brain MRI Analysis |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2512.18679 |