Pixel-Wise Recognition for Holistic Surgical Scene Understanding
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915079300055040 |
|---|---|
| author | Ayobi, Nicolás Rodríguez, Santiago Pérez, Alejandra Hernández, Isabela Aparicio, Nicolás Dessevres, Eugénie Peña, Sebastián Santander, Jessica Caicedo, Juan Ignacio Fernández, Nicolás Arbeláez, Pablo |
| author_facet | Ayobi, Nicolás Rodríguez, Santiago Pérez, Alejandra Hernández, Isabela Aparicio, Nicolás Dessevres, Eugénie Peña, Sebastián Santander, Jessica Caicedo, Juan Ignacio Fernández, Nicolás Arbeláez, Pablo |
| contents | This paper presents the Holistic and Multi-Granular Surgical Scene Understanding of Prostatectomies (GraSP) dataset, a curated benchmark that models surgical scene understanding as a hierarchy of complementary tasks with varying levels of granularity. Our approach encompasses long-term tasks, such as surgical phase and step recognition, and short-term tasks, including surgical instrument segmentation and atomic visual actions detection. To exploit our proposed benchmark, we introduce the Transformers for Actions, Phases, Steps, and Instrument Segmentation (TAPIS) model, a general architecture that combines a global video feature extractor with localized region proposals from an instrument segmentation model to tackle the multi-granularity of our benchmark. Through extensive experimentation in ours and alternative benchmarks, we demonstrate TAPIS's versatility and state-of-the-art performance across different tasks. This work represents a foundational step forward in Endoscopic Vision, offering a novel framework for future research towards holistic surgical scene understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_11174 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pixel-Wise Recognition for Holistic Surgical Scene Understanding Ayobi, Nicolás Rodríguez, Santiago Pérez, Alejandra Hernández, Isabela Aparicio, Nicolás Dessevres, Eugénie Peña, Sebastián Santander, Jessica Caicedo, Juan Ignacio Fernández, Nicolás Arbeláez, Pablo Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning This paper presents the Holistic and Multi-Granular Surgical Scene Understanding of Prostatectomies (GraSP) dataset, a curated benchmark that models surgical scene understanding as a hierarchy of complementary tasks with varying levels of granularity. Our approach encompasses long-term tasks, such as surgical phase and step recognition, and short-term tasks, including surgical instrument segmentation and atomic visual actions detection. To exploit our proposed benchmark, we introduce the Transformers for Actions, Phases, Steps, and Instrument Segmentation (TAPIS) model, a general architecture that combines a global video feature extractor with localized region proposals from an instrument segmentation model to tackle the multi-granularity of our benchmark. Through extensive experimentation in ours and alternative benchmarks, we demonstrate TAPIS's versatility and state-of-the-art performance across different tasks. This work represents a foundational step forward in Endoscopic Vision, offering a novel framework for future research towards holistic surgical scene understanding. |
| title | Pixel-Wise Recognition for Holistic Surgical Scene Understanding |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2401.11174 |