End to End AI System for Surgical Gesture Sequence Recognition and Clinical Outcome Prediction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909903646359552 |
|---|---|
| author | Li, Xi Matsumoto, Nicholas Pasupulety, Ujjwal Deo, Atharva Yang, Cherine Moran, Jay Hernandez, Miguel E. Wager, Peter Lin, Jasmine Kim, Jeanine Goh, Alvin C. Wagner, Christian Sonn, Geoffrey A. Hung, Andrew J. |
| author_facet | Li, Xi Matsumoto, Nicholas Pasupulety, Ujjwal Deo, Atharva Yang, Cherine Moran, Jay Hernandez, Miguel E. Wager, Peter Lin, Jasmine Kim, Jeanine Goh, Alvin C. Wagner, Christian Sonn, Geoffrey A. Hung, Andrew J. |
| contents | Fine-grained analysis of intraoperative behavior and its impact on patient outcomes remain a longstanding challenge. We present Frame-to-Outcome (F2O), an end-to-end system that translates tissue dissection videos into gesture sequences and uncovers patterns associated with postoperative outcomes. Leveraging transformer-based spatial and temporal modeling and frame-wise classification, F2O robustly detects consecutive short (~2 seconds) gestures in the nerve-sparing step of robot-assisted radical prostatectomy (AUC: 0.80 frame-level; 0.81 video-level). F2O-derived features (gesture frequency, duration, and transitions) predicted postoperative outcomes with accuracy comparable to human annotations (0.79 vs. 0.75; overlapping 95% CI). Across 25 shared features, effect size directions were concordant with small differences (~ 0.07), and strong correlation (r = 0.96, p < 1e-14). F2O also captured key patterns linked to erectile function recovery, including prolonged tissue peeling and reduced energy use. By enabling automatic interpretable assessment, F2O establishes a foundation for data-driven surgical feedback and prospective clinical decision support. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_11899 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | End to End AI System for Surgical Gesture Sequence Recognition and Clinical Outcome Prediction Li, Xi Matsumoto, Nicholas Pasupulety, Ujjwal Deo, Atharva Yang, Cherine Moran, Jay Hernandez, Miguel E. Wager, Peter Lin, Jasmine Kim, Jeanine Goh, Alvin C. Wagner, Christian Sonn, Geoffrey A. Hung, Andrew J. Artificial Intelligence Computer Vision and Pattern Recognition Fine-grained analysis of intraoperative behavior and its impact on patient outcomes remain a longstanding challenge. We present Frame-to-Outcome (F2O), an end-to-end system that translates tissue dissection videos into gesture sequences and uncovers patterns associated with postoperative outcomes. Leveraging transformer-based spatial and temporal modeling and frame-wise classification, F2O robustly detects consecutive short (~2 seconds) gestures in the nerve-sparing step of robot-assisted radical prostatectomy (AUC: 0.80 frame-level; 0.81 video-level). F2O-derived features (gesture frequency, duration, and transitions) predicted postoperative outcomes with accuracy comparable to human annotations (0.79 vs. 0.75; overlapping 95% CI). Across 25 shared features, effect size directions were concordant with small differences (~ 0.07), and strong correlation (r = 0.96, p < 1e-14). F2O also captured key patterns linked to erectile function recovery, including prolonged tissue peeling and reduced energy use. By enabling automatic interpretable assessment, F2O establishes a foundation for data-driven surgical feedback and prospective clinical decision support. |
| title | End to End AI System for Surgical Gesture Sequence Recognition and Clinical Outcome Prediction |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.11899 |