DexWild: Dexterous Human Interactions for In-the-Wild Robot Policies
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909050403291136 |
|---|---|
| author | Tao, Tony Srirama, Mohan Kumar Liu, Jason Jingzhou Shaw, Kenneth Pathak, Deepak |
| author_facet | Tao, Tony Srirama, Mohan Kumar Liu, Jason Jingzhou Shaw, Kenneth Pathak, Deepak |
| contents | Large-scale, diverse robot datasets have emerged as a promising path toward enabling dexterous manipulation policies to generalize to novel environments, but acquiring such datasets presents many challenges. While teleoperation provides high-fidelity datasets, its high cost limits its scalability. Instead, what if people could use their own hands, just as they do in everyday life, to collect data? In DexWild, a diverse team of data collectors uses their hands to collect hours of interactions across a multitude of environments and objects. To record this data, we create DexWild-System, a low-cost, mobile, and easy-to-use device. The DexWild learning framework co-trains on both human and robot demonstrations, leading to improved performance compared to training on each dataset individually. This combination results in robust robot policies capable of generalizing to novel environments, tasks, and embodiments with minimal additional robot-specific data. Experimental results demonstrate that DexWild significantly improves performance, achieving a 68.5% success rate in unseen environments-nearly four times higher than policies trained with robot data only-and offering 5.8x better cross-embodiment generalization. Video results, codebases, and instructions at https://dexwild.github.io |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_07813 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DexWild: Dexterous Human Interactions for In-the-Wild Robot Policies Tao, Tony Srirama, Mohan Kumar Liu, Jason Jingzhou Shaw, Kenneth Pathak, Deepak Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Systems and Control Large-scale, diverse robot datasets have emerged as a promising path toward enabling dexterous manipulation policies to generalize to novel environments, but acquiring such datasets presents many challenges. While teleoperation provides high-fidelity datasets, its high cost limits its scalability. Instead, what if people could use their own hands, just as they do in everyday life, to collect data? In DexWild, a diverse team of data collectors uses their hands to collect hours of interactions across a multitude of environments and objects. To record this data, we create DexWild-System, a low-cost, mobile, and easy-to-use device. The DexWild learning framework co-trains on both human and robot demonstrations, leading to improved performance compared to training on each dataset individually. This combination results in robust robot policies capable of generalizing to novel environments, tasks, and embodiments with minimal additional robot-specific data. Experimental results demonstrate that DexWild significantly improves performance, achieving a 68.5% success rate in unseen environments-nearly four times higher than policies trained with robot data only-and offering 5.8x better cross-embodiment generalization. Video results, codebases, and instructions at https://dexwild.github.io |
| title | DexWild: Dexterous Human Interactions for In-the-Wild Robot Policies |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Systems and Control |
| url | https://arxiv.org/abs/2505.07813 |