Endo-CLIP: Progressive Self-Supervised Pre-training on Raw Colonoscopy Records
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908363387830272 |
|---|---|
| author | He, Yili Zhu, Yan Fu, Peiyao Yang, Ruijie Chen, Tianyi Wang, Zhihua Li, Quanlin Zhou, Pinghong Yang, Xian Wang, Shuo |
| author_facet | He, Yili Zhu, Yan Fu, Peiyao Yang, Ruijie Chen, Tianyi Wang, Zhihua Li, Quanlin Zhou, Pinghong Yang, Xian Wang, Shuo |
| contents | Pre-training on image-text colonoscopy records offers substantial potential for improving endoscopic image analysis, but faces challenges including non-informative background images, complex medical terminology, and ambiguous multi-lesion descriptions. We introduce Endo-CLIP, a novel self-supervised framework that enhances Contrastive Language-Image Pre-training (CLIP) for this domain. Endo-CLIP's three-stage framework--cleansing, attunement, and unification--addresses these challenges by (1) removing background frames, (2) leveraging large language models to extract clinical attributes for fine-grained contrastive learning, and (3) employing patient-level cross-attention to resolve multi-polyp ambiguities. Extensive experiments demonstrate that Endo-CLIP significantly outperforms state-of-the-art pre-training methods in zero-shot and few-shot polyp detection and classification, paving the way for more accurate and clinically relevant endoscopic analysis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_09435 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Endo-CLIP: Progressive Self-Supervised Pre-training on Raw Colonoscopy Records He, Yili Zhu, Yan Fu, Peiyao Yang, Ruijie Chen, Tianyi Wang, Zhihua Li, Quanlin Zhou, Pinghong Yang, Xian Wang, Shuo Computer Vision and Pattern Recognition Artificial Intelligence Pre-training on image-text colonoscopy records offers substantial potential for improving endoscopic image analysis, but faces challenges including non-informative background images, complex medical terminology, and ambiguous multi-lesion descriptions. We introduce Endo-CLIP, a novel self-supervised framework that enhances Contrastive Language-Image Pre-training (CLIP) for this domain. Endo-CLIP's three-stage framework--cleansing, attunement, and unification--addresses these challenges by (1) removing background frames, (2) leveraging large language models to extract clinical attributes for fine-grained contrastive learning, and (3) employing patient-level cross-attention to resolve multi-polyp ambiguities. Extensive experiments demonstrate that Endo-CLIP significantly outperforms state-of-the-art pre-training methods in zero-shot and few-shot polyp detection and classification, paving the way for more accurate and clinically relevant endoscopic analysis. |
| title | Endo-CLIP: Progressive Self-Supervised Pre-training on Raw Colonoscopy Records |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2505.09435 |