MIRAGE: Multimodal foundation model and benchmark for comprehensive retinal OCT image analysis
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909814274129920 |
|---|---|
| author | Morano, José Fazekas, Botond Sükei, Emese Fecso, Ronald Emre, Taha Gumpinger, Markus Faustmann, Georg Oghbaie, Marzieh Schmidt-Erfurth, Ursula Bogunović, Hrvoje |
| author_facet | Morano, José Fazekas, Botond Sükei, Emese Fecso, Ronald Emre, Taha Gumpinger, Markus Faustmann, Georg Oghbaie, Marzieh Schmidt-Erfurth, Ursula Bogunović, Hrvoje |
| contents | Artificial intelligence (AI) has become a fundamental tool for assisting clinicians in analyzing ophthalmic images, such as optical coherence tomography (OCT). However, developing AI models often requires extensive annotation, and existing models tend to underperform on independent, unseen data. Foundation models (FMs), large AI models trained on vast unlabeled datasets, have shown promise in overcoming these challenges. Nonetheless, available FMs for ophthalmology lack extensive validation, especially for segmentation tasks, and focus on a single imaging modality. In this context, we propose MIRAGE, a novel multimodal FM for the analysis of OCT and scanning laser ophthalmoscopy (SLO) images. Additionally, we propose a new evaluation benchmark with OCT/SLO classification and segmentation tasks. The comparison with general and specialized FMs and segmentation methods shows the superiority of MIRAGE in both types of tasks, highlighting its suitability as a basis for the development of robust AI systems for retinal OCT image analysis. Both MIRAGE and the evaluation benchmark are publicly available: https://github.com/j-morano/MIRAGE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_08900 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MIRAGE: Multimodal foundation model and benchmark for comprehensive retinal OCT image analysis Morano, José Fazekas, Botond Sükei, Emese Fecso, Ronald Emre, Taha Gumpinger, Markus Faustmann, Georg Oghbaie, Marzieh Schmidt-Erfurth, Ursula Bogunović, Hrvoje Computer Vision and Pattern Recognition Artificial intelligence (AI) has become a fundamental tool for assisting clinicians in analyzing ophthalmic images, such as optical coherence tomography (OCT). However, developing AI models often requires extensive annotation, and existing models tend to underperform on independent, unseen data. Foundation models (FMs), large AI models trained on vast unlabeled datasets, have shown promise in overcoming these challenges. Nonetheless, available FMs for ophthalmology lack extensive validation, especially for segmentation tasks, and focus on a single imaging modality. In this context, we propose MIRAGE, a novel multimodal FM for the analysis of OCT and scanning laser ophthalmoscopy (SLO) images. Additionally, we propose a new evaluation benchmark with OCT/SLO classification and segmentation tasks. The comparison with general and specialized FMs and segmentation methods shows the superiority of MIRAGE in both types of tasks, highlighting its suitability as a basis for the development of robust AI systems for retinal OCT image analysis. Both MIRAGE and the evaluation benchmark are publicly available: https://github.com/j-morano/MIRAGE. |
| title | MIRAGE: Multimodal foundation model and benchmark for comprehensive retinal OCT image analysis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.08900 |