Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.17934 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914287122907136 |
|---|---|
| author | Vu, Vi Nguyen, Thanh-Huy Nguyen, Tien-Thinh Lam, Ba-Thinh Nguyen, Hoang-Thien Wang, Tianyang Li, Xingjian Xu, Min |
| author_facet | Vu, Vi Nguyen, Thanh-Huy Nguyen, Tien-Thinh Lam, Ba-Thinh Nguyen, Hoang-Thien Wang, Tianyang Li, Xingjian Xu, Min |
| contents | Foundation models like the Segment Anything Model (SAM) show strong generalization, yet adapting them to medical images remains difficult due to domain shift, scarce labels, and the inability of Parameter-Efficient Fine-Tuning (PEFT) to exploit unlabeled data. While conventional models like U-Net excel in semi-supervised medical learning, their potential to assist a PEFT SAM has been largely overlooked. We introduce SC-SAM, a specialist-generalist framework where U-Net provides point-based prompts and pseudo-labels to guide SAM's adaptation, while SAM serves as a powerful generalist supervisor to regularize U-Net. This reciprocal guidance forms a bidirectional co-training loop that allows both models to effectively exploit the unlabeled data. Across prostate MRI and polyp segmentation benchmarks, our method achieves state-of-the-art results, outperforming other existing semi-supervised SAM variants and even medical foundation models like MedSAM, highlighting the value of specialist-generalist cooperation for label-efficient medical image segmentation. Our code is available at https://github.com/vnlvi2k3/SC-SAM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_17934 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | From Specialist to Generalist: Unlocking SAM's Learning Potential on Unlabeled Medical Images Vu, Vi Nguyen, Thanh-Huy Nguyen, Tien-Thinh Lam, Ba-Thinh Nguyen, Hoang-Thien Wang, Tianyang Li, Xingjian Xu, Min Computer Vision and Pattern Recognition Artificial Intelligence Foundation models like the Segment Anything Model (SAM) show strong generalization, yet adapting them to medical images remains difficult due to domain shift, scarce labels, and the inability of Parameter-Efficient Fine-Tuning (PEFT) to exploit unlabeled data. While conventional models like U-Net excel in semi-supervised medical learning, their potential to assist a PEFT SAM has been largely overlooked. We introduce SC-SAM, a specialist-generalist framework where U-Net provides point-based prompts and pseudo-labels to guide SAM's adaptation, while SAM serves as a powerful generalist supervisor to regularize U-Net. This reciprocal guidance forms a bidirectional co-training loop that allows both models to effectively exploit the unlabeled data. Across prostate MRI and polyp segmentation benchmarks, our method achieves state-of-the-art results, outperforming other existing semi-supervised SAM variants and even medical foundation models like MedSAM, highlighting the value of specialist-generalist cooperation for label-efficient medical image segmentation. Our code is available at https://github.com/vnlvi2k3/SC-SAM. |
| title | From Specialist to Generalist: Unlocking SAM's Learning Potential on Unlabeled Medical Images |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2601.17934 |