Adapting a Segmentation Foundation Model for Medical Image Classification
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913828863737856 |
|---|---|
| author | Gu, Pengfei Tang, Haoteng Ebeid, Islam A. Nunez, Jose A. Vazquez, Fabian Adame, Diego Zhan, Marcus Li, Huimin Fu, Bin Chen, Danny Z. |
| author_facet | Gu, Pengfei Tang, Haoteng Ebeid, Islam A. Nunez, Jose A. Vazquez, Fabian Adame, Diego Zhan, Marcus Li, Huimin Fu, Bin Chen, Danny Z. |
| contents | Recent advancements in foundation models, such as the Segment Anything Model (SAM), have shown strong performance in various vision tasks, particularly image segmentation, due to their impressive zero-shot segmentation capabilities. However, effectively adapting such models for medical image classification is still a less explored topic. In this paper, we introduce a new framework to adapt SAM for medical image classification. First, we utilize the SAM image encoder as a feature extractor to capture segmentation-based features that convey important spatial and contextual details of the image, while freezing its weights to avoid unnecessary overhead during training. Next, we propose a novel Spatially Localized Channel Attention (SLCA) mechanism to compute spatially localized attention weights for the feature maps. The features extracted from SAM's image encoder are processed through SLCA to compute attention weights, which are then integrated into deep learning classification models to enhance their focus on spatially relevant or meaningful regions of the image, thus improving classification performance. Experimental results on three public medical image classification datasets demonstrate the effectiveness and data-efficiency of our approach. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_06217 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Adapting a Segmentation Foundation Model for Medical Image Classification Gu, Pengfei Tang, Haoteng Ebeid, Islam A. Nunez, Jose A. Vazquez, Fabian Adame, Diego Zhan, Marcus Li, Huimin Fu, Bin Chen, Danny Z. Computer Vision and Pattern Recognition Recent advancements in foundation models, such as the Segment Anything Model (SAM), have shown strong performance in various vision tasks, particularly image segmentation, due to their impressive zero-shot segmentation capabilities. However, effectively adapting such models for medical image classification is still a less explored topic. In this paper, we introduce a new framework to adapt SAM for medical image classification. First, we utilize the SAM image encoder as a feature extractor to capture segmentation-based features that convey important spatial and contextual details of the image, while freezing its weights to avoid unnecessary overhead during training. Next, we propose a novel Spatially Localized Channel Attention (SLCA) mechanism to compute spatially localized attention weights for the feature maps. The features extracted from SAM's image encoder are processed through SLCA to compute attention weights, which are then integrated into deep learning classification models to enhance their focus on spatially relevant or meaningful regions of the image, thus improving classification performance. Experimental results on three public medical image classification datasets demonstrate the effectiveness and data-efficiency of our approach. |
| title | Adapting a Segmentation Foundation Model for Medical Image Classification |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.06217 |