ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914071528341504 |
|---|---|
| author | Zeng, Qiyuan Li, Chengmeng John, Jude St. Zhou, Zhongyi Wen, Junjie Feng, Guorui Zhu, Yichen Xu, Yi |
| author_facet | Zeng, Qiyuan Li, Chengmeng John, Jude St. Zhou, Zhongyi Wen, Junjie Feng, Guorui Zhu, Yichen Xu, Yi |
| contents | We present ActiveUMI, a framework for a data collection system that transfers in-the-wild human demonstrations to robots capable of complex bimanual manipulation. ActiveUMI couples a portable VR teleoperation kit with sensorized controllers that mirror the robot's end-effectors, bridging human-robot kinematics via precise pose alignment. To ensure mobility and data quality, we introduce several key techniques, including immersive 3D model rendering, a self-contained wearable computer, and efficient calibration methods. ActiveUMI's defining feature is its capture of active, egocentric perception. By recording an operator's deliberate head movements via a head-mounted display, our system learns the crucial link between visual attention and manipulation. We evaluate ActiveUMI on six challenging bimanual tasks. Policies trained exclusively on ActiveUMI data achieve an average success rate of 70\% on in-distribution tasks and demonstrate strong generalization, retaining a 56\% success rate when tested on novel objects and in new environments. Our results demonstrate that portable data collection systems, when coupled with learned active perception, provide an effective and scalable pathway toward creating generalizable and highly capable real-world robot policies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_01607 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrations Zeng, Qiyuan Li, Chengmeng John, Jude St. Zhou, Zhongyi Wen, Junjie Feng, Guorui Zhu, Yichen Xu, Yi Robotics Computer Vision and Pattern Recognition We present ActiveUMI, a framework for a data collection system that transfers in-the-wild human demonstrations to robots capable of complex bimanual manipulation. ActiveUMI couples a portable VR teleoperation kit with sensorized controllers that mirror the robot's end-effectors, bridging human-robot kinematics via precise pose alignment. To ensure mobility and data quality, we introduce several key techniques, including immersive 3D model rendering, a self-contained wearable computer, and efficient calibration methods. ActiveUMI's defining feature is its capture of active, egocentric perception. By recording an operator's deliberate head movements via a head-mounted display, our system learns the crucial link between visual attention and manipulation. We evaluate ActiveUMI on six challenging bimanual tasks. Policies trained exclusively on ActiveUMI data achieve an average success rate of 70\% on in-distribution tasks and demonstrate strong generalization, retaining a 56\% success rate when tested on novel objects and in new environments. Our results demonstrate that portable data collection systems, when coupled with learned active perception, provide an effective and scalable pathway toward creating generalizable and highly capable real-world robot policies. |
| title | ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrations |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.01607 |