Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.21017 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918472688074752 |
|---|---|
| author | Consortium, Open-H-Embodiment : Nelson, Nigel Chen, Juo-Tung Haworth, Jesse Chen, Xinhao Zbinden, Lukas Huang, Dianye Abdelaal, Alaa Eldin Arezzo, Alberto Acar, Ayberk Alambeigi, Farshid Ammirati, Carlo Alberto Ao, Yunke Rodriguez, Pablo David Aranda Atar, Soofiyan Ballo, Mattia Barnes, Noah Barontini, Federica Binkiewicz, Filip Black, Peter Bodenstedt, Sebastian Borgioli, Leonardo Budjak, Nikola Calmé, Benjamin Carrillo, Fabio Cavalcanti, Nicola Chen, Changwei Chen, Haoxin Chen, Sihang Chen, Qihan Chen, Zhongyu Chen, Ziyang Cheng, Shing Shin Cheng, Meiqing Cheng, Min Chiu, Zih-Yun Sarah Chu, Xiangyu Correa-Gallego, Camilo Dagnino, Giulio Deguet, Anton Delgado, Jacob DeLong, Jonathan C. Deng, Kaizhong Dimitrakakis, Alexander Ding, Qingpeng Ding, Hao Distefano, Giovanni Donoho, Daniel Duan, Anqing Esposito, Marco Farritor, Shane Fayad, Jad Fayad, Zahi Ferradosa, Mario Filicori, Filippo Finn, Chelsea Fürnstahl, Philipp Ge, Jiawei Giannarou, Stamatia Ludevid, Xavier Giralt Giraud, Frederic Godbole, Aditya Amit Goldberg, Ken Goldenberg, Antony Marana, Diego Granero Guo, Xiaoqing Haidegger, Tamás Hailey, Evan Hansen, Pascal Hao, Ziyi Hari, Kush Hayashi, Kengo Hawkins, Jonathon Haworth, Shelby Hellig, Ortrun Herrell, S. Duke Hong, Zhouyang Howe, Andrew Hu, Junlei Hu, Zhaoyang Jacopo Jain, Ria Javazm, Mohammad Rafiee Ji, Howard Ji, Rui Ji, Jianmin Jiang, Zhongliang Jones, Dominic Jopling, Jeffrey Jordan, Britton Ju, Ran Kam, Michael Kang, Luoyao Kang, Fausto Kapuria, Siddhartha Kazanzides, Peter Kiehler, Sonika Kilmer, Ethan Kim, Ji Woong Korzeniowski, Przemysław Kuchi, Chandra Kumar, Nithesh Kuntz, Alan Lavagno, Federico Lee, Yu Chung Lee, Hao-Chih Li, Hang Li, Zhen Liang, Xiao Lin, Xinxin Lin, Jinsong Liu, Chang Liu, Fei Liu, Pei Liu, Yun-hui Liuchen, Wanli Lukács, Eszter Mann, Sareena Mannas, Miles Marinelli, Brett Martyniak, Sabina Marzola, Francesco Mazza, Lorenzo Mei, Xueyan Morais, Maria Clara Muratore, Luigi Narayanaswamy, Chetan Reddy Naskręt, Michał Navarro-Alarcon, David Neary, Cyrus Ng, Chi Kit Nguan, Christopher Noonan, David Oh, Ki Hwan Olesch, Tom Christian Okamura, Allison M. Opfermann, Justin Pescio, Matteo Pham, Doan Xuan Viet Porras, Tito Ren, Hongliang Jimenez, Ariel Rodriguez Baena, Ferdinando Rodriguez y Salcudean, Septimiu E. Sathya, Asmitha Satish, Preethi Seenivasan, Lalithkumar Shao, Jiaqi Shen, Yiqing Sheng, Yu Shi, Lucy XiaoYang Soulé, Zoe Speidel, Stefanie Su, Mingwu Su, Jianhao Sunmola, Idris Takács, Kristóf Tang, Yunxi Thornycroft, Patrick Tian, Yu Thompson, Jordan Turkcan, Mehmet K. Unberath, Mathias Valdastri, Pietro Vives, Carlos Vuong, Quan Wagner, Martin Wang, Farong Wang, Wei Wang, Lidian Wang, Chung-Pang Wang, Guankun Wang, Junyi Wang, Erqi Wang, Ziyi Watts, Tanner Wein, Wolfgang Wu, Yimeng Wu, Zijian Wu, Hongjun Wu, Luohong Wu, Jie Ying Wu, Junlin Wu, Victoria Wu, Kaixuan Wójcikowski, Mateusz Xiao, Yunye Xiao, Nan Xie, Wenxuan Yang, Hao Yang, Tianqi Yang, Yinuo Ye, Menglong Yeung, Ryan S. Yilmaz, Nural Yin, Chim Ho Yip, Michael Younis, Rayan Yu, Chenhao Zaman, Sayem Nazmuz Zefran, Milos Zhang, Han Zhang, Yuelin Zhang, Yidong Zhang, Yanyong Zhang, Xuyang Zhang, Yameng Zhang, Joyce Zhong, Ning Zhou, Peng Zhou, Haoying Zuo, Xiuli Navab, Nassir Azizian, Mahdi Huver, Sean D. Krieger, Axel |
| author_facet | Consortium, Open-H-Embodiment : Nelson, Nigel Chen, Juo-Tung Haworth, Jesse Chen, Xinhao Zbinden, Lukas Huang, Dianye Abdelaal, Alaa Eldin Arezzo, Alberto Acar, Ayberk Alambeigi, Farshid Ammirati, Carlo Alberto Ao, Yunke Rodriguez, Pablo David Aranda Atar, Soofiyan Ballo, Mattia Barnes, Noah Barontini, Federica Binkiewicz, Filip Black, Peter Bodenstedt, Sebastian Borgioli, Leonardo Budjak, Nikola Calmé, Benjamin Carrillo, Fabio Cavalcanti, Nicola Chen, Changwei Chen, Haoxin Chen, Sihang Chen, Qihan Chen, Zhongyu Chen, Ziyang Cheng, Shing Shin Cheng, Meiqing Cheng, Min Chiu, Zih-Yun Sarah Chu, Xiangyu Correa-Gallego, Camilo Dagnino, Giulio Deguet, Anton Delgado, Jacob DeLong, Jonathan C. Deng, Kaizhong Dimitrakakis, Alexander Ding, Qingpeng Ding, Hao Distefano, Giovanni Donoho, Daniel Duan, Anqing Esposito, Marco Farritor, Shane Fayad, Jad Fayad, Zahi Ferradosa, Mario Filicori, Filippo Finn, Chelsea Fürnstahl, Philipp Ge, Jiawei Giannarou, Stamatia Ludevid, Xavier Giralt Giraud, Frederic Godbole, Aditya Amit Goldberg, Ken Goldenberg, Antony Marana, Diego Granero Guo, Xiaoqing Haidegger, Tamás Hailey, Evan Hansen, Pascal Hao, Ziyi Hari, Kush Hayashi, Kengo Hawkins, Jonathon Haworth, Shelby Hellig, Ortrun Herrell, S. Duke Hong, Zhouyang Howe, Andrew Hu, Junlei Hu, Zhaoyang Jacopo Jain, Ria Javazm, Mohammad Rafiee Ji, Howard Ji, Rui Ji, Jianmin Jiang, Zhongliang Jones, Dominic Jopling, Jeffrey Jordan, Britton Ju, Ran Kam, Michael Kang, Luoyao Kang, Fausto Kapuria, Siddhartha Kazanzides, Peter Kiehler, Sonika Kilmer, Ethan Kim, Ji Woong Korzeniowski, Przemysław Kuchi, Chandra Kumar, Nithesh Kuntz, Alan Lavagno, Federico Lee, Yu Chung Lee, Hao-Chih Li, Hang Li, Zhen Liang, Xiao Lin, Xinxin Lin, Jinsong Liu, Chang Liu, Fei Liu, Pei Liu, Yun-hui Liuchen, Wanli Lukács, Eszter Mann, Sareena Mannas, Miles Marinelli, Brett Martyniak, Sabina Marzola, Francesco Mazza, Lorenzo Mei, Xueyan Morais, Maria Clara Muratore, Luigi Narayanaswamy, Chetan Reddy Naskręt, Michał Navarro-Alarcon, David Neary, Cyrus Ng, Chi Kit Nguan, Christopher Noonan, David Oh, Ki Hwan Olesch, Tom Christian Okamura, Allison M. Opfermann, Justin Pescio, Matteo Pham, Doan Xuan Viet Porras, Tito Ren, Hongliang Jimenez, Ariel Rodriguez Baena, Ferdinando Rodriguez y Salcudean, Septimiu E. Sathya, Asmitha Satish, Preethi Seenivasan, Lalithkumar Shao, Jiaqi Shen, Yiqing Sheng, Yu Shi, Lucy XiaoYang Soulé, Zoe Speidel, Stefanie Su, Mingwu Su, Jianhao Sunmola, Idris Takács, Kristóf Tang, Yunxi Thornycroft, Patrick Tian, Yu Thompson, Jordan Turkcan, Mehmet K. Unberath, Mathias Valdastri, Pietro Vives, Carlos Vuong, Quan Wagner, Martin Wang, Farong Wang, Wei Wang, Lidian Wang, Chung-Pang Wang, Guankun Wang, Junyi Wang, Erqi Wang, Ziyi Watts, Tanner Wein, Wolfgang Wu, Yimeng Wu, Zijian Wu, Hongjun Wu, Luohong Wu, Jie Ying Wu, Junlin Wu, Victoria Wu, Kaixuan Wójcikowski, Mateusz Xiao, Yunye Xiao, Nan Xie, Wenxuan Yang, Hao Yang, Tianqi Yang, Yinuo Ye, Menglong Yeung, Ryan S. Yilmaz, Nural Yin, Chim Ho Yip, Michael Younis, Rayan Yu, Chenhao Zaman, Sayem Nazmuz Zefran, Milos Zhang, Han Zhang, Yuelin Zhang, Yidong Zhang, Yanyong Zhang, Xuyang Zhang, Yameng Zhang, Joyce Zhong, Ning Zhou, Peng Zhou, Haoying Zuo, Xiuli Navab, Nassir Azizian, Mahdi Huver, Sean D. Krieger, Axel |
| contents | Autonomous medical robots hold promise to improve patient outcomes, reduce provider workload, democratize access to care, and enable superhuman precision. However, autonomous medical robotics has been limited by a fundamental data problem: existing medical robotic datasets are small, single-embodiment, and rarely shared openly, restricting the development of foundation models that the field needs to advance. We introduce Open-H-Embodiment, the largest open dataset of medical robotic video with synchronized kinematics to date, spanning more than 49 institutions and multiple robotic platforms including the CMR Versius, Intuitive Surgical's da Vinci, da Vinci Research Kit (dVRK), Rob Surgical BiTrack, Virtual Incision's MIRA, Moon Surgical Maestro, and a variety of custom systems, spanning surgical manipulation, robotic ultrasound, and endoscopy procedures. We demonstrate the research enabled by this dataset through two foundation models. GR00T-H is the first open foundation vision-language-action model for medical robotics, which is the only evaluated model to achieve full end-to-end task completion on a structured suturing benchmark (25% of trials vs. 0% for all others) and achieves 64% average success across a 29-step ex vivo suturing sequence. We also train Cosmos-H-Surgical-Simulator, the first action-conditioned world model to enable multi-embodiment surgical simulation from a single checkpoint, spanning nine robotic platforms and supporting in silico policy evaluation and synthetic data generation for the medical domain. These results suggest that open, large-scale medical robot data collection can serve as critical infrastructure for the research community, enabling advances in robot learning, world modeling, and beyond. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_21017 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics Consortium, Open-H-Embodiment : Nelson, Nigel Chen, Juo-Tung Haworth, Jesse Chen, Xinhao Zbinden, Lukas Huang, Dianye Abdelaal, Alaa Eldin Arezzo, Alberto Acar, Ayberk Alambeigi, Farshid Ammirati, Carlo Alberto Ao, Yunke Rodriguez, Pablo David Aranda Atar, Soofiyan Ballo, Mattia Barnes, Noah Barontini, Federica Binkiewicz, Filip Black, Peter Bodenstedt, Sebastian Borgioli, Leonardo Budjak, Nikola Calmé, Benjamin Carrillo, Fabio Cavalcanti, Nicola Chen, Changwei Chen, Haoxin Chen, Sihang Chen, Qihan Chen, Zhongyu Chen, Ziyang Cheng, Shing Shin Cheng, Meiqing Cheng, Min Chiu, Zih-Yun Sarah Chu, Xiangyu Correa-Gallego, Camilo Dagnino, Giulio Deguet, Anton Delgado, Jacob DeLong, Jonathan C. Deng, Kaizhong Dimitrakakis, Alexander Ding, Qingpeng Ding, Hao Distefano, Giovanni Donoho, Daniel Duan, Anqing Esposito, Marco Farritor, Shane Fayad, Jad Fayad, Zahi Ferradosa, Mario Filicori, Filippo Finn, Chelsea Fürnstahl, Philipp Ge, Jiawei Giannarou, Stamatia Ludevid, Xavier Giralt Giraud, Frederic Godbole, Aditya Amit Goldberg, Ken Goldenberg, Antony Marana, Diego Granero Guo, Xiaoqing Haidegger, Tamás Hailey, Evan Hansen, Pascal Hao, Ziyi Hari, Kush Hayashi, Kengo Hawkins, Jonathon Haworth, Shelby Hellig, Ortrun Herrell, S. Duke Hong, Zhouyang Howe, Andrew Hu, Junlei Hu, Zhaoyang Jacopo Jain, Ria Javazm, Mohammad Rafiee Ji, Howard Ji, Rui Ji, Jianmin Jiang, Zhongliang Jones, Dominic Jopling, Jeffrey Jordan, Britton Ju, Ran Kam, Michael Kang, Luoyao Kang, Fausto Kapuria, Siddhartha Kazanzides, Peter Kiehler, Sonika Kilmer, Ethan Kim, Ji Woong Korzeniowski, Przemysław Kuchi, Chandra Kumar, Nithesh Kuntz, Alan Lavagno, Federico Lee, Yu Chung Lee, Hao-Chih Li, Hang Li, Zhen Liang, Xiao Lin, Xinxin Lin, Jinsong Liu, Chang Liu, Fei Liu, Pei Liu, Yun-hui Liuchen, Wanli Lukács, Eszter Mann, Sareena Mannas, Miles Marinelli, Brett Martyniak, Sabina Marzola, Francesco Mazza, Lorenzo Mei, Xueyan Morais, Maria Clara Muratore, Luigi Narayanaswamy, Chetan Reddy Naskręt, Michał Navarro-Alarcon, David Neary, Cyrus Ng, Chi Kit Nguan, Christopher Noonan, David Oh, Ki Hwan Olesch, Tom Christian Okamura, Allison M. Opfermann, Justin Pescio, Matteo Pham, Doan Xuan Viet Porras, Tito Ren, Hongliang Jimenez, Ariel Rodriguez Baena, Ferdinando Rodriguez y Salcudean, Septimiu E. Sathya, Asmitha Satish, Preethi Seenivasan, Lalithkumar Shao, Jiaqi Shen, Yiqing Sheng, Yu Shi, Lucy XiaoYang Soulé, Zoe Speidel, Stefanie Su, Mingwu Su, Jianhao Sunmola, Idris Takács, Kristóf Tang, Yunxi Thornycroft, Patrick Tian, Yu Thompson, Jordan Turkcan, Mehmet K. Unberath, Mathias Valdastri, Pietro Vives, Carlos Vuong, Quan Wagner, Martin Wang, Farong Wang, Wei Wang, Lidian Wang, Chung-Pang Wang, Guankun Wang, Junyi Wang, Erqi Wang, Ziyi Watts, Tanner Wein, Wolfgang Wu, Yimeng Wu, Zijian Wu, Hongjun Wu, Luohong Wu, Jie Ying Wu, Junlin Wu, Victoria Wu, Kaixuan Wójcikowski, Mateusz Xiao, Yunye Xiao, Nan Xie, Wenxuan Yang, Hao Yang, Tianqi Yang, Yinuo Ye, Menglong Yeung, Ryan S. Yilmaz, Nural Yin, Chim Ho Yip, Michael Younis, Rayan Yu, Chenhao Zaman, Sayem Nazmuz Zefran, Milos Zhang, Han Zhang, Yuelin Zhang, Yidong Zhang, Yanyong Zhang, Xuyang Zhang, Yameng Zhang, Joyce Zhong, Ning Zhou, Peng Zhou, Haoying Zuo, Xiuli Navab, Nassir Azizian, Mahdi Huver, Sean D. Krieger, Axel Robotics Artificial Intelligence Autonomous medical robots hold promise to improve patient outcomes, reduce provider workload, democratize access to care, and enable superhuman precision. However, autonomous medical robotics has been limited by a fundamental data problem: existing medical robotic datasets are small, single-embodiment, and rarely shared openly, restricting the development of foundation models that the field needs to advance. We introduce Open-H-Embodiment, the largest open dataset of medical robotic video with synchronized kinematics to date, spanning more than 49 institutions and multiple robotic platforms including the CMR Versius, Intuitive Surgical's da Vinci, da Vinci Research Kit (dVRK), Rob Surgical BiTrack, Virtual Incision's MIRA, Moon Surgical Maestro, and a variety of custom systems, spanning surgical manipulation, robotic ultrasound, and endoscopy procedures. We demonstrate the research enabled by this dataset through two foundation models. GR00T-H is the first open foundation vision-language-action model for medical robotics, which is the only evaluated model to achieve full end-to-end task completion on a structured suturing benchmark (25% of trials vs. 0% for all others) and achieves 64% average success across a 29-step ex vivo suturing sequence. We also train Cosmos-H-Surgical-Simulator, the first action-conditioned world model to enable multi-embodiment surgical simulation from a single checkpoint, spanning nine robotic platforms and supporting in silico policy evaluation and synthetic data generation for the medical domain. These results suggest that open, large-scale medical robot data collection can serve as critical infrastructure for the research community, enabling advances in robot learning, world modeling, and beyond. |
| title | Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2604.21017 |