Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917607451394048 |
|---|---|
| author | He, Tairan Luo, Zhengyi Xiao, Wenli Zhang, Chong Kitani, Kris Liu, Changliu Shi, Guanya |
| author_facet | He, Tairan Luo, Zhengyi Xiao, Wenli Zhang, Chong Kitani, Kris Liu, Changliu Shi, Guanya |
| contents | We present Human to Humanoid (H2O), a reinforcement learning (RL) based framework that enables real-time whole-body teleoperation of a full-sized humanoid robot with only an RGB camera. To create a large-scale retargeted motion dataset of human movements for humanoid robots, we propose a scalable "sim-to-data" process to filter and pick feasible motions using a privileged motion imitator. Afterwards, we train a robust real-time humanoid motion imitator in simulation using these refined motions and transfer it to the real humanoid robot in a zero-shot manner. We successfully achieve teleoperation of dynamic whole-body motions in real-world scenarios, including walking, back jumping, kicking, turning, waving, pushing, boxing, etc. To the best of our knowledge, this is the first demonstration to achieve learning-based real-time whole-body humanoid teleoperation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_04436 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation He, Tairan Luo, Zhengyi Xiao, Wenli Zhang, Chong Kitani, Kris Liu, Changliu Shi, Guanya Robotics Artificial Intelligence Machine Learning Systems and Control We present Human to Humanoid (H2O), a reinforcement learning (RL) based framework that enables real-time whole-body teleoperation of a full-sized humanoid robot with only an RGB camera. To create a large-scale retargeted motion dataset of human movements for humanoid robots, we propose a scalable "sim-to-data" process to filter and pick feasible motions using a privileged motion imitator. Afterwards, we train a robust real-time humanoid motion imitator in simulation using these refined motions and transfer it to the real humanoid robot in a zero-shot manner. We successfully achieve teleoperation of dynamic whole-body motions in real-world scenarios, including walking, back jumping, kicking, turning, waving, pushing, boxing, etc. To the best of our knowledge, this is the first demonstration to achieve learning-based real-time whole-body humanoid teleoperation. |
| title | Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation |
| topic | Robotics Artificial Intelligence Machine Learning Systems and Control |
| url | https://arxiv.org/abs/2403.04436 |