A Foundation Model for Brain Lesion Segmentation with Mixture of Modality Experts
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909256346763264 |
|---|---|
| author | Zhang, Xinru Ou, Ni Basaran, Berke Doga Visentin, Marco Qiao, Mengyun Gu, Renyang Ouyang, Cheng Liu, Yaou Matthew, Paul M. Ye, Chuyang Bai, Wenjia |
| author_facet | Zhang, Xinru Ou, Ni Basaran, Berke Doga Visentin, Marco Qiao, Mengyun Gu, Renyang Ouyang, Cheng Liu, Yaou Matthew, Paul M. Ye, Chuyang Bai, Wenjia |
| contents | Brain lesion segmentation plays an essential role in neurological research and diagnosis. As brain lesions can be caused by various pathological alterations, different types of brain lesions tend to manifest with different characteristics on different imaging modalities. Due to this complexity, brain lesion segmentation methods are often developed in a task-specific manner. A specific segmentation model is developed for a particular lesion type and imaging modality. However, the use of task-specific models requires predetermination of the lesion type and imaging modality, which complicates their deployment in real-world scenarios. In this work, we propose a universal foundation model for 3D brain lesion segmentation, which can automatically segment different types of brain lesions for input data of various imaging modalities. We formulate a novel Mixture of Modality Experts (MoME) framework with multiple expert networks attending to different imaging modalities. A hierarchical gating network combines the expert predictions and fosters expertise collaboration. Furthermore, we introduce a curriculum learning strategy during training to avoid the degeneration of each expert network and preserve their specialization. We evaluated the proposed method on nine brain lesion datasets, encompassing five imaging modalities and eight lesion types. The results show that our model outperforms state-of-the-art universal models and provides promising generalization to unseen datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_10246 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Foundation Model for Brain Lesion Segmentation with Mixture of Modality Experts Zhang, Xinru Ou, Ni Basaran, Berke Doga Visentin, Marco Qiao, Mengyun Gu, Renyang Ouyang, Cheng Liu, Yaou Matthew, Paul M. Ye, Chuyang Bai, Wenjia Image and Video Processing Computer Vision and Pattern Recognition Brain lesion segmentation plays an essential role in neurological research and diagnosis. As brain lesions can be caused by various pathological alterations, different types of brain lesions tend to manifest with different characteristics on different imaging modalities. Due to this complexity, brain lesion segmentation methods are often developed in a task-specific manner. A specific segmentation model is developed for a particular lesion type and imaging modality. However, the use of task-specific models requires predetermination of the lesion type and imaging modality, which complicates their deployment in real-world scenarios. In this work, we propose a universal foundation model for 3D brain lesion segmentation, which can automatically segment different types of brain lesions for input data of various imaging modalities. We formulate a novel Mixture of Modality Experts (MoME) framework with multiple expert networks attending to different imaging modalities. A hierarchical gating network combines the expert predictions and fosters expertise collaboration. Furthermore, we introduce a curriculum learning strategy during training to avoid the degeneration of each expert network and preserve their specialization. We evaluated the proposed method on nine brain lesion datasets, encompassing five imaging modalities and eight lesion types. The results show that our model outperforms state-of-the-art universal models and provides promising generalization to unseen datasets. |
| title | A Foundation Model for Brain Lesion Segmentation with Mixture of Modality Experts |
| topic | Image and Video Processing Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.10246 |