Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2306.10518 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914801200922624 |
|---|---|
| author | Xu, Shusheng Wang, Huaijie Gao, Jiaxuan Ouyang, Yutao Yu, Chao Wu, Yi |
| author_facet | Xu, Shusheng Wang, Huaijie Gao, Jiaxuan Ouyang, Yutao Yu, Chao Wu, Yi |
| contents | We aim to control a robot to physically behave in the real world following any high-level language command like "cartwheel" or "kick". Although human motion datasets exist, this task remains particularly challenging since generative models can produce physically unrealistic motions, which will be more severe for robots due to different body structures and physical properties. Deploying such a motion to a physical robot can cause even greater difficulties due to the sim2real gap. We develop LAnguage-Guided mOtion cONtrol (LAGOON), a multi-phase reinforcement learning (RL) method to generate physically realistic robot motions under language commands. LAGOON first leverages a pretrained model to generate a human motion from a language command. Then an RL phase trains a control policy in simulation to mimic the generated human motion. Finally, with domain randomization, our learned policy can be deployed to a quadrupedal robot, leading to a quadrupedal robot that can take diverse behaviors in the real world under natural language commands |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_10518 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | LAGOON: Language-Guided Motion Control Xu, Shusheng Wang, Huaijie Gao, Jiaxuan Ouyang, Yutao Yu, Chao Wu, Yi Robotics We aim to control a robot to physically behave in the real world following any high-level language command like "cartwheel" or "kick". Although human motion datasets exist, this task remains particularly challenging since generative models can produce physically unrealistic motions, which will be more severe for robots due to different body structures and physical properties. Deploying such a motion to a physical robot can cause even greater difficulties due to the sim2real gap. We develop LAnguage-Guided mOtion cONtrol (LAGOON), a multi-phase reinforcement learning (RL) method to generate physically realistic robot motions under language commands. LAGOON first leverages a pretrained model to generate a human motion from a language command. Then an RL phase trains a control policy in simulation to mimic the generated human motion. Finally, with domain randomization, our learned policy can be deployed to a quadrupedal robot, leading to a quadrupedal robot that can take diverse behaviors in the real world under natural language commands |
| title | LAGOON: Language-Guided Motion Control |
| topic | Robotics |
| url | https://arxiv.org/abs/2306.10518 |