SurgSync: Time-Synchronized Multi-Modal Data Collection Framework and Dataset for Surgical Robotics
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908871455408128 |
|---|---|
| author | Zhou, Haoying Liu, Chang Wu, Yimeng Wu, Junlin Wu, Zijian Lee, Yu Chung Martuscelli, Sara Salcudean, Spetimiu E. Fischer, Gregory S. Kazanzides, Peter |
| author_facet | Zhou, Haoying Liu, Chang Wu, Yimeng Wu, Junlin Wu, Zijian Lee, Yu Chung Martuscelli, Sara Salcudean, Spetimiu E. Fischer, Gregory S. Kazanzides, Peter |
| contents | Most existing robotic surgery systems adopt a human-in-the-loop paradigm, often with the surgeon directly teleoperating the robotic system. Adding intelligence to these robots would enable higher-level control, such as supervised autonomy or even full autonomy. However, artificial intelligence (AI) requires large amounts of training data, which is currently lacking. This work proposes SurgSync, a multi-modal data collection framework with offline and online synchronization to support training and real-time inference, respectively. The framework is implemented on a da Vinci Research Kit (dVRK) and introduces (1) dual-mode (online/offline-matching) synchronized recorders, (2) a modern stereo endoscope to achieve image quality on par with clinical systems, and (3) additional sensors such as a side-view camera and a novel capacitive contact sensor to provide ground truth contact data. The framework also incorporates a post-processing toolbox for tasks such as depth estimation, optical flow, and a practical kinematic reprojection method using Gaussian heatmap. User studies with participants of varying skill levels are performed with ex-vivo tissue to provide clinically realistic data, and a network for surgical skill assessment is employed to demonstrate utilization of the collected data. Through the user study experiments, we obtained a dataset of 214 validated instances across multiple canonical training tasks. All software and data are available at surgsync.github.io. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_06919 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SurgSync: Time-Synchronized Multi-Modal Data Collection Framework and Dataset for Surgical Robotics Zhou, Haoying Liu, Chang Wu, Yimeng Wu, Junlin Wu, Zijian Lee, Yu Chung Martuscelli, Sara Salcudean, Spetimiu E. Fischer, Gregory S. Kazanzides, Peter Robotics Most existing robotic surgery systems adopt a human-in-the-loop paradigm, often with the surgeon directly teleoperating the robotic system. Adding intelligence to these robots would enable higher-level control, such as supervised autonomy or even full autonomy. However, artificial intelligence (AI) requires large amounts of training data, which is currently lacking. This work proposes SurgSync, a multi-modal data collection framework with offline and online synchronization to support training and real-time inference, respectively. The framework is implemented on a da Vinci Research Kit (dVRK) and introduces (1) dual-mode (online/offline-matching) synchronized recorders, (2) a modern stereo endoscope to achieve image quality on par with clinical systems, and (3) additional sensors such as a side-view camera and a novel capacitive contact sensor to provide ground truth contact data. The framework also incorporates a post-processing toolbox for tasks such as depth estimation, optical flow, and a practical kinematic reprojection method using Gaussian heatmap. User studies with participants of varying skill levels are performed with ex-vivo tissue to provide clinically realistic data, and a network for surgical skill assessment is employed to demonstrate utilization of the collected data. Through the user study experiments, we obtained a dataset of 214 validated instances across multiple canonical training tasks. All software and data are available at surgsync.github.io. |
| title | SurgSync: Time-Synchronized Multi-Modal Data Collection Framework and Dataset for Surgical Robotics |
| topic | Robotics |
| url | https://arxiv.org/abs/2603.06919 |