Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.08464 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913541801377792 |
|---|---|
| author | Chen, Sirui Wang, Chen Nguyen, Kaden Fei-Fei, Li Liu, C. Karen |
| author_facet | Chen, Sirui Wang, Chen Nguyen, Kaden Fei-Fei, Li Liu, C. Karen |
| contents | Recent progress in imitation learning from human demonstrations has shown promising results in teaching robots manipulation skills. To further scale up training datasets, recent works start to use portable data collection devices without the need for physical robot hardware. However, due to the absence of on-robot feedback during data collection, the data quality depends heavily on user expertise, and many devices are limited to specific robot embodiments. We propose ARCap, a portable data collection system that provides visual feedback through augmented reality (AR) and haptic warnings to guide users in collecting high-quality demonstrations. Through extensive user studies, we show that ARCap enables novice users to collect robot-executable data that matches robot kinematics and avoids collisions with the scenes. With data collected from ARCap, robots can perform challenging tasks, such as manipulation in cluttered environments and long-horizon cross-embodiment manipulation. ARCap is fully open-source and easy to calibrate; all components are built from off-the-shelf products. More details and results can be found on our website: https://stanford-tml.github.io/ARCap |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_08464 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ARCap: Collecting High-quality Human Demonstrations for Robot Learning with Augmented Reality Feedback Chen, Sirui Wang, Chen Nguyen, Kaden Fei-Fei, Li Liu, C. Karen Robotics Artificial Intelligence Recent progress in imitation learning from human demonstrations has shown promising results in teaching robots manipulation skills. To further scale up training datasets, recent works start to use portable data collection devices without the need for physical robot hardware. However, due to the absence of on-robot feedback during data collection, the data quality depends heavily on user expertise, and many devices are limited to specific robot embodiments. We propose ARCap, a portable data collection system that provides visual feedback through augmented reality (AR) and haptic warnings to guide users in collecting high-quality demonstrations. Through extensive user studies, we show that ARCap enables novice users to collect robot-executable data that matches robot kinematics and avoids collisions with the scenes. With data collected from ARCap, robots can perform challenging tasks, such as manipulation in cluttered environments and long-horizon cross-embodiment manipulation. ARCap is fully open-source and easy to calibrate; all components are built from off-the-shelf products. More details and results can be found on our website: https://stanford-tml.github.io/ARCap |
| title | ARCap: Collecting High-quality Human Demonstrations for Robot Learning with Augmented Reality Feedback |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2410.08464 |