Affordance-Based Disambiguation of Surgical Instructions for Collaborative Robot-Assisted Surgery
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918144049676288 |
|---|---|
| author | Davila, Ana Colan, Jacinto Hasegawa, Yasuhisa |
| author_facet | Davila, Ana Colan, Jacinto Hasegawa, Yasuhisa |
| contents | Effective human-robot collaboration in surgery is affected by the inherent ambiguity of verbal communication. This paper presents a framework for a robotic surgical assistant that interprets and disambiguates verbal instructions from a surgeon by grounding them in the visual context of the operating field. The system employs a two-level affordance-based reasoning process that first analyzes the surgical scene using a multimodal vision-language model and then reasons about the instruction using a knowledge base of tool capabilities. To ensure patient safety, a dual-set conformal prediction method is used to provide a statistically rigorous confidence measure for robot decisions, allowing it to identify and flag ambiguous commands. We evaluated our framework on a curated dataset of ambiguous surgical requests from cholecystectomy videos, demonstrating a general disambiguation rate of 60% and presenting a method for safer human-robot interaction in the operating room. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_14967 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Affordance-Based Disambiguation of Surgical Instructions for Collaborative Robot-Assisted Surgery Davila, Ana Colan, Jacinto Hasegawa, Yasuhisa Robotics Human-Computer Interaction Effective human-robot collaboration in surgery is affected by the inherent ambiguity of verbal communication. This paper presents a framework for a robotic surgical assistant that interprets and disambiguates verbal instructions from a surgeon by grounding them in the visual context of the operating field. The system employs a two-level affordance-based reasoning process that first analyzes the surgical scene using a multimodal vision-language model and then reasons about the instruction using a knowledge base of tool capabilities. To ensure patient safety, a dual-set conformal prediction method is used to provide a statistically rigorous confidence measure for robot decisions, allowing it to identify and flag ambiguous commands. We evaluated our framework on a curated dataset of ambiguous surgical requests from cholecystectomy videos, demonstrating a general disambiguation rate of 60% and presenting a method for safer human-robot interaction in the operating room. |
| title | Affordance-Based Disambiguation of Surgical Instructions for Collaborative Robot-Assisted Surgery |
| topic | Robotics Human-Computer Interaction |
| url | https://arxiv.org/abs/2509.14967 |