Scalable Dexterous Robot Learning with AR-based Remote Human-Robot Interactions
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918326695886848 |
|---|---|
| author | Yang, Yicheng Li, Ruijiao Wang, Lifeng Zheng, Shuai Ma, Shunzheng Zhang, Keyu Sun, Tuoyu Dai, Chenyun Ding, Jie Zou, Zhuo |
| author_facet | Yang, Yicheng Li, Ruijiao Wang, Lifeng Zheng, Shuai Ma, Shunzheng Zhang, Keyu Sun, Tuoyu Dai, Chenyun Ding, Jie Zou, Zhuo |
| contents | This paper focuses on the scalable robot learning for manipulation in the dexterous robot arm-hand systems, where the remote human-robot interactions via augmented reality (AR) are established to collect the expert demonstration data for improving efficiency. In such a system, we present a unified framework to address the general manipulation task problem. Specifically, the proposed method consists of two phases: i) In the first phase for pretraining, the policy is created in a behavior cloning (BC) manner, through leveraging the learning data from our AR-based remote human-robot interaction system; ii) In the second phase, a contrastive learning empowered reinforcement learning (RL) method is developed to obtain more efficient and robust policy than the BC, and thus a projection head is designed to accelerate the learning progress. An event-driven augmented reward is adopted for enhancing the safety. To validate the proposed method, both the physics simulations via PyBullet and real-world experiments are carried out. The results demonstrate that compared to the classic proximal policy optimization and soft actor-critic policies, our method not only significantly speeds up the inference, but also achieves much better performance in terms of the success rate for fulfilling the manipulation tasks. By conducting the ablation study, it is confirmed that the proposed RL with contrastive learning overcomes policy collapse. Supplementary demonstrations are available at https://cyberyyc.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_07341 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Scalable Dexterous Robot Learning with AR-based Remote Human-Robot Interactions Yang, Yicheng Li, Ruijiao Wang, Lifeng Zheng, Shuai Ma, Shunzheng Zhang, Keyu Sun, Tuoyu Dai, Chenyun Ding, Jie Zou, Zhuo Machine Learning Robotics This paper focuses on the scalable robot learning for manipulation in the dexterous robot arm-hand systems, where the remote human-robot interactions via augmented reality (AR) are established to collect the expert demonstration data for improving efficiency. In such a system, we present a unified framework to address the general manipulation task problem. Specifically, the proposed method consists of two phases: i) In the first phase for pretraining, the policy is created in a behavior cloning (BC) manner, through leveraging the learning data from our AR-based remote human-robot interaction system; ii) In the second phase, a contrastive learning empowered reinforcement learning (RL) method is developed to obtain more efficient and robust policy than the BC, and thus a projection head is designed to accelerate the learning progress. An event-driven augmented reward is adopted for enhancing the safety. To validate the proposed method, both the physics simulations via PyBullet and real-world experiments are carried out. The results demonstrate that compared to the classic proximal policy optimization and soft actor-critic policies, our method not only significantly speeds up the inference, but also achieves much better performance in terms of the success rate for fulfilling the manipulation tasks. By conducting the ablation study, it is confirmed that the proposed RL with contrastive learning overcomes policy collapse. Supplementary demonstrations are available at https://cyberyyc.github.io/. |
| title | Scalable Dexterous Robot Learning with AR-based Remote Human-Robot Interactions |
| topic | Machine Learning Robotics |
| url | https://arxiv.org/abs/2602.07341 |