Segment Anything Model for Medical Images?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913198371766272 |
|---|---|
| author | Huang, Yuhao Yang, Xin Liu, Lian Zhou, Han Chang, Ao Zhou, Xinrui Chen, Rusi Yu, Junxuan Chen, Jiongquan Chen, Chaoyu Liu, Sijing Chi, Haozhe Hu, Xindi Yue, Kejuan Li, Lei Grau, Vicente Fan, Deng-Ping Dong, Fajin Ni, Dong |
| author_facet | Huang, Yuhao Yang, Xin Liu, Lian Zhou, Han Chang, Ao Zhou, Xinrui Chen, Rusi Yu, Junxuan Chen, Jiongquan Chen, Chaoyu Liu, Sijing Chi, Haozhe Hu, Xindi Yue, Kejuan Li, Lei Grau, Vicente Fan, Deng-Ping Dong, Fajin Ni, Dong |
| contents | The Segment Anything Model (SAM) is the first foundation model for general image segmentation. It has achieved impressive results on various natural image segmentation tasks. However, medical image segmentation (MIS) is more challenging because of the complex modalities, fine anatomical structures, uncertain and complex object boundaries, and wide-range object scales. To fully validate SAM's performance on medical data, we collected and sorted 53 open-source datasets and built a large medical segmentation dataset with 18 modalities, 84 objects, 125 object-modality paired targets, 1050K 2D images, and 6033K masks. We comprehensively analyzed different models and strategies on the so-called COSMOS 1050K dataset. Our findings mainly include the following: 1) SAM showed remarkable performance in some specific objects but was unstable, imperfect, or even totally failed in other situations. 2) SAM with the large ViT-H showed better overall performance than that with the small ViT-B. 3) SAM performed better with manual hints, especially box, than the Everything mode. 4) SAM could help human annotation with high labeling quality and less time. 5) SAM was sensitive to the randomness in the center point and tight box prompts, and may suffer from a serious performance drop. 6) SAM performed better than interactive methods with one or a few points, but will be outpaced as the number of points increases. 7) SAM's performance correlated to different factors, including boundary complexity, intensity differences, etc. 8) Finetuning the SAM on specific medical tasks could improve its average DICE performance by 4.39% and 6.68% for ViT-B and ViT-H, respectively. We hope that this comprehensive report can help researchers explore the potential of SAM applications in MIS, and guide how to appropriately use and develop SAM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_14660 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Segment Anything Model for Medical Images? Huang, Yuhao Yang, Xin Liu, Lian Zhou, Han Chang, Ao Zhou, Xinrui Chen, Rusi Yu, Junxuan Chen, Jiongquan Chen, Chaoyu Liu, Sijing Chi, Haozhe Hu, Xindi Yue, Kejuan Li, Lei Grau, Vicente Fan, Deng-Ping Dong, Fajin Ni, Dong Image and Video Processing Computer Vision and Pattern Recognition Machine Learning The Segment Anything Model (SAM) is the first foundation model for general image segmentation. It has achieved impressive results on various natural image segmentation tasks. However, medical image segmentation (MIS) is more challenging because of the complex modalities, fine anatomical structures, uncertain and complex object boundaries, and wide-range object scales. To fully validate SAM's performance on medical data, we collected and sorted 53 open-source datasets and built a large medical segmentation dataset with 18 modalities, 84 objects, 125 object-modality paired targets, 1050K 2D images, and 6033K masks. We comprehensively analyzed different models and strategies on the so-called COSMOS 1050K dataset. Our findings mainly include the following: 1) SAM showed remarkable performance in some specific objects but was unstable, imperfect, or even totally failed in other situations. 2) SAM with the large ViT-H showed better overall performance than that with the small ViT-B. 3) SAM performed better with manual hints, especially box, than the Everything mode. 4) SAM could help human annotation with high labeling quality and less time. 5) SAM was sensitive to the randomness in the center point and tight box prompts, and may suffer from a serious performance drop. 6) SAM performed better than interactive methods with one or a few points, but will be outpaced as the number of points increases. 7) SAM's performance correlated to different factors, including boundary complexity, intensity differences, etc. 8) Finetuning the SAM on specific medical tasks could improve its average DICE performance by 4.39% and 6.68% for ViT-B and ViT-H, respectively. We hope that this comprehensive report can help researchers explore the potential of SAM applications in MIS, and guide how to appropriately use and develop SAM. |
| title | Segment Anything Model for Medical Images? |
| topic | Image and Video Processing Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2304.14660 |