Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915226270564352 |
|---|---|
| author | Rau, Anita Endo, Mark Aklilu, Josiah Heo, Jaewoo Saab, Khaled Paderno, Alberto Jopling, Jeffrey Holsinger, F. Christopher Yeung-Levy, Serena |
| author_facet | Rau, Anita Endo, Mark Aklilu, Josiah Heo, Jaewoo Saab, Khaled Paderno, Alberto Jopling, Jeffrey Holsinger, F. Christopher Yeung-Levy, Serena |
| contents | Large Vision-Language Models offer a new paradigm for AI-driven image understanding, enabling models to perform tasks without task-specific training. This flexibility holds particular promise across medicine, where expert-annotated data is scarce. Yet, VLMs' practical utility in intervention-focused domains--especially surgery, where decision-making is subjective and clinical scenarios are variable--remains uncertain. Here, we present a comprehensive analysis of 11 state-of-the-art VLMs across 17 key visual understanding tasks in surgical AI--from anatomy recognition to skill assessment--using 13 datasets spanning laparoscopic, robotic, and open procedures. In our experiments, VLMs demonstrate promising generalizability, at times outperforming supervised models when deployed outside their training setting. In-context learning, incorporating examples during testing, boosted performance up to three-fold, suggesting adaptability as a key strength. Still, tasks requiring spatial or temporal reasoning remained difficult. Beyond surgery, our findings offer insights into VLMs' potential for tackling complex and dynamic scenarios in clinical and broader real-world applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_02799 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence Rau, Anita Endo, Mark Aklilu, Josiah Heo, Jaewoo Saab, Khaled Paderno, Alberto Jopling, Jeffrey Holsinger, F. Christopher Yeung-Levy, Serena Computer Vision and Pattern Recognition Artificial Intelligence Large Vision-Language Models offer a new paradigm for AI-driven image understanding, enabling models to perform tasks without task-specific training. This flexibility holds particular promise across medicine, where expert-annotated data is scarce. Yet, VLMs' practical utility in intervention-focused domains--especially surgery, where decision-making is subjective and clinical scenarios are variable--remains uncertain. Here, we present a comprehensive analysis of 11 state-of-the-art VLMs across 17 key visual understanding tasks in surgical AI--from anatomy recognition to skill assessment--using 13 datasets spanning laparoscopic, robotic, and open procedures. In our experiments, VLMs demonstrate promising generalizability, at times outperforming supervised models when deployed outside their training setting. In-context learning, incorporating examples during testing, boosted performance up to three-fold, suggesting adaptability as a key strength. Still, tasks requiring spatial or temporal reasoning remained difficult. Beyond surgery, our findings offer insights into VLMs' potential for tackling complex and dynamic scenarios in clinical and broader real-world applications. |
| title | Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2504.02799 |