Towards Holistic Surgical Scene Understanding
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2022
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911764737687552 |
|---|---|
| author | Valderrama, Natalia Puentes, Paola Ruiz Hernández, Isabela Ayobi, Nicolás Verlyk, Mathilde Santander, Jessica Caicedo, Juan Fernández, Nicolás Arbeláez, Pablo |
| author_facet | Valderrama, Natalia Puentes, Paola Ruiz Hernández, Isabela Ayobi, Nicolás Verlyk, Mathilde Santander, Jessica Caicedo, Juan Fernández, Nicolás Arbeláez, Pablo |
| contents | Most benchmarks for studying surgical interventions focus on a specific challenge instead of leveraging the intrinsic complementarity among different tasks. In this work, we present a new experimental framework towards holistic surgical scene understanding. First, we introduce the Phase, Step, Instrument, and Atomic Visual Action recognition (PSI-AVA) Dataset. PSI-AVA includes annotations for both long-term (Phase and Step recognition) and short-term reasoning (Instrument detection and novel Atomic Action recognition) in robot-assisted radical prostatectomy videos. Second, we present Transformers for Action, Phase, Instrument, and steps Recognition (TAPIR) as a strong baseline for surgical scene understanding. TAPIR leverages our dataset's multi-level annotations as it benefits from the learned representation on the instrument detection task to improve its classification capacity. Our experimental results in both PSI-AVA and other publicly available databases demonstrate the adequacy of our framework to spur future research on holistic surgical scene understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2212_04582 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Towards Holistic Surgical Scene Understanding Valderrama, Natalia Puentes, Paola Ruiz Hernández, Isabela Ayobi, Nicolás Verlyk, Mathilde Santander, Jessica Caicedo, Juan Fernández, Nicolás Arbeláez, Pablo Computer Vision and Pattern Recognition Artificial Intelligence Most benchmarks for studying surgical interventions focus on a specific challenge instead of leveraging the intrinsic complementarity among different tasks. In this work, we present a new experimental framework towards holistic surgical scene understanding. First, we introduce the Phase, Step, Instrument, and Atomic Visual Action recognition (PSI-AVA) Dataset. PSI-AVA includes annotations for both long-term (Phase and Step recognition) and short-term reasoning (Instrument detection and novel Atomic Action recognition) in robot-assisted radical prostatectomy videos. Second, we present Transformers for Action, Phase, Instrument, and steps Recognition (TAPIR) as a strong baseline for surgical scene understanding. TAPIR leverages our dataset's multi-level annotations as it benefits from the learned representation on the instrument detection task to improve its classification capacity. Our experimental results in both PSI-AVA and other publicly available databases demonstrate the adequacy of our framework to spur future research on holistic surgical scene understanding. |
| title | Towards Holistic Surgical Scene Understanding |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2212.04582 |