Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910166060892160 |
|---|---|
| author | Kang, Jingxuan Zhang, Ziqi Zheng, Shaoming Li, Shuang Patel, Uday Bharat Fitzhugh, Alexander Harry Lung, Phillip Kiberu, Yusuf Jathanna, Nikesh Jamil-Copley, Shahnaz Kainz, Bernhard Qin, Chen |
| author_facet | Kang, Jingxuan Zhang, Ziqi Zheng, Shaoming Li, Shuang Patel, Uday Bharat Fitzhugh, Alexander Harry Lung, Phillip Kiberu, Yusuf Jathanna, Nikesh Jamil-Copley, Shahnaz Kainz, Bernhard Qin, Chen |
| contents | Segmentation is central to clinical diagnosis and monitoring, yet the reliability of modern foundation models in medical imaging still depends on the availability of precise prompts. The Segment Anything Model (SAM) offers powerful zero-shot capabilities, although it collapses under the weak, generic, and noisy prompts that dominate real clinical workflows. In practice, annotations such as centerline points are coarse and ambiguous, often drifting across neighboring anatomy and misguiding SAM toward inconsistent or incomplete masks. We introduce SPD, a Saliency-Guided Prompt Distillation framework that converts these unreliable cues into robust guidance. SPD first learns data-driven anatomical priors through a lightweight saliency head to obtain confident localization maps. These priors then drive Contextual Prompt Distillation, which validates and enriches noisy prompts using cues from anatomically adjacent slices, producing a consensus prompt set that matches the behavior of expert reasoning. A Pairwise Slice Consistency objective further enforces local anatomical coherence during segmentation. Experiments on four challenging MRI and CT benchmarks demonstrate that SPD consistently outperforms existing SAM adaptations and supervised baselines, delivering large gains in both region-based and boundary-based metrics. SPD provides a practical and principled path toward reliable foundation model deployment in clinical environments where only imperfect prompts are available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_23314 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM Kang, Jingxuan Zhang, Ziqi Zheng, Shaoming Li, Shuang Patel, Uday Bharat Fitzhugh, Alexander Harry Lung, Phillip Kiberu, Yusuf Jathanna, Nikesh Jamil-Copley, Shahnaz Kainz, Bernhard Qin, Chen Computer Vision and Pattern Recognition Segmentation is central to clinical diagnosis and monitoring, yet the reliability of modern foundation models in medical imaging still depends on the availability of precise prompts. The Segment Anything Model (SAM) offers powerful zero-shot capabilities, although it collapses under the weak, generic, and noisy prompts that dominate real clinical workflows. In practice, annotations such as centerline points are coarse and ambiguous, often drifting across neighboring anatomy and misguiding SAM toward inconsistent or incomplete masks. We introduce SPD, a Saliency-Guided Prompt Distillation framework that converts these unreliable cues into robust guidance. SPD first learns data-driven anatomical priors through a lightweight saliency head to obtain confident localization maps. These priors then drive Contextual Prompt Distillation, which validates and enriches noisy prompts using cues from anatomically adjacent slices, producing a consensus prompt set that matches the behavior of expert reasoning. A Pairwise Slice Consistency objective further enforces local anatomical coherence during segmentation. Experiments on four challenging MRI and CT benchmarks demonstrate that SPD consistently outperforms existing SAM adaptations and supervised baselines, delivering large gains in both region-based and boundary-based metrics. SPD provides a practical and principled path toward reliable foundation model deployment in clinical environments where only imperfect prompts are available. |
| title | Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.23314 |