Open-TeleVision: Teleoperation with Immersive Active Visual Feedback
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917714661998592 |
|---|---|
| author | Cheng, Xuxin Li, Jialong Yang, Shiqi Yang, Ge Wang, Xiaolong |
| author_facet | Cheng, Xuxin Li, Jialong Yang, Shiqi Yang, Ge Wang, Xiaolong |
| contents | Teleoperation serves as a powerful method for collecting on-robot data essential for robot learning from demonstrations. The intuitiveness and ease of use of the teleoperation system are crucial for ensuring high-quality, diverse, and scalable data. To achieve this, we propose an immersive teleoperation system Open-TeleVision that allows operators to actively perceive the robot's surroundings in a stereoscopic manner. Additionally, the system mirrors the operator's arm and hand movements on the robot, creating an immersive experience as if the operator's mind is transmitted to a robot embodiment. We validate the effectiveness of our system by collecting data and training imitation learning policies on four long-horizon, precise tasks (Can Sorting, Can Insertion, Folding, and Unloading) for 2 different humanoid robots and deploy them in the real world. The system is open-sourced at: https://robot-tv.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_01512 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Open-TeleVision: Teleoperation with Immersive Active Visual Feedback Cheng, Xuxin Li, Jialong Yang, Shiqi Yang, Ge Wang, Xiaolong Robotics Human-Computer Interaction Machine Learning Teleoperation serves as a powerful method for collecting on-robot data essential for robot learning from demonstrations. The intuitiveness and ease of use of the teleoperation system are crucial for ensuring high-quality, diverse, and scalable data. To achieve this, we propose an immersive teleoperation system Open-TeleVision that allows operators to actively perceive the robot's surroundings in a stereoscopic manner. Additionally, the system mirrors the operator's arm and hand movements on the robot, creating an immersive experience as if the operator's mind is transmitted to a robot embodiment. We validate the effectiveness of our system by collecting data and training imitation learning policies on four long-horizon, precise tasks (Can Sorting, Can Insertion, Folding, and Unloading) for 2 different humanoid robots and deploy them in the real world. The system is open-sourced at: https://robot-tv.github.io/ |
| title | Open-TeleVision: Teleoperation with Immersive Active Visual Feedback |
| topic | Robotics Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2407.01512 |