DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915529936076800 |
|---|---|
| author | Xu, Mengda Zhang, Han Hou, Yifan Xu, Zhenjia Fan, Linxi Veloso, Manuela Song, Shuran |
| author_facet | Xu, Mengda Zhang, Han Hou, Yifan Xu, Zhenjia Fan, Linxi Veloso, Manuela Song, Shuran |
| contents | We present DexUMI - a data collection and policy learning framework that uses the human hand as the natural interface to transfer dexterous manipulation skills to various robot hands. DexUMI includes hardware and software adaptations to minimize the embodiment gap between the human hand and various robot hands. The hardware adaptation bridges the kinematics gap using a wearable hand exoskeleton. It allows direct haptic feedback in manipulation data collection and adapts human motion to feasible robot hand motion. The software adaptation bridges the visual gap by replacing the human hand in video data with high-fidelity robot hand inpainting. We demonstrate DexUMI's capabilities through comprehensive real-world experiments on two different dexterous robot hand hardware platforms, achieving an average task success rate of 86%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_21864 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation Xu, Mengda Zhang, Han Hou, Yifan Xu, Zhenjia Fan, Linxi Veloso, Manuela Song, Shuran Robotics We present DexUMI - a data collection and policy learning framework that uses the human hand as the natural interface to transfer dexterous manipulation skills to various robot hands. DexUMI includes hardware and software adaptations to minimize the embodiment gap between the human hand and various robot hands. The hardware adaptation bridges the kinematics gap using a wearable hand exoskeleton. It allows direct haptic feedback in manipulation data collection and adapts human motion to feasible robot hand motion. The software adaptation bridges the visual gap by replacing the human hand in video data with high-fidelity robot hand inpainting. We demonstrate DexUMI's capabilities through comprehensive real-world experiments on two different dexterous robot hand hardware platforms, achieving an average task success rate of 86%. |
| title | DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation |
| topic | Robotics |
| url | https://arxiv.org/abs/2505.21864 |