Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Control
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916646699925504 |
|---|---|
| author | Lu, Chenhao Cheng, Xuxin Li, Jialong Yang, Shiqi Ji, Mazeyu Yuan, Chengjing Yang, Ge Yi, Sha Wang, Xiaolong |
| author_facet | Lu, Chenhao Cheng, Xuxin Li, Jialong Yang, Shiqi Ji, Mazeyu Yuan, Chengjing Yang, Ge Yi, Sha Wang, Xiaolong |
| contents | Humanoid robots require both robust lower-body locomotion and precise upper-body manipulation. While recent Reinforcement Learning (RL) approaches provide whole-body loco-manipulation policies, they lack precise manipulation with high DoF arms. In this paper, we propose decoupling upper-body control from locomotion, using inverse kinematics (IK) and motion retargeting for precise manipulation, while RL focuses on robust lower-body locomotion. We introduce PMP (Predictive Motion Priors), trained with Conditional Variational Autoencoder (CVAE) to effectively represent upper-body motions. The locomotion policy is trained conditioned on this upper-body motion representation, ensuring that the system remains robust with both manipulation and locomotion. We show that CVAE features are crucial for stability and robustness, and significantly outperforms RL-based whole-body control in precise manipulation. With precise upper-body motion and robust lower-body locomotion control, operators can remotely control the humanoid to walk around and explore different environments, while performing diverse manipulation tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_07773 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Control Lu, Chenhao Cheng, Xuxin Li, Jialong Yang, Shiqi Ji, Mazeyu Yuan, Chengjing Yang, Ge Yi, Sha Wang, Xiaolong Robotics Artificial Intelligence Machine Learning Humanoid robots require both robust lower-body locomotion and precise upper-body manipulation. While recent Reinforcement Learning (RL) approaches provide whole-body loco-manipulation policies, they lack precise manipulation with high DoF arms. In this paper, we propose decoupling upper-body control from locomotion, using inverse kinematics (IK) and motion retargeting for precise manipulation, while RL focuses on robust lower-body locomotion. We introduce PMP (Predictive Motion Priors), trained with Conditional Variational Autoencoder (CVAE) to effectively represent upper-body motions. The locomotion policy is trained conditioned on this upper-body motion representation, ensuring that the system remains robust with both manipulation and locomotion. We show that CVAE features are crucial for stability and robustness, and significantly outperforms RL-based whole-body control in precise manipulation. With precise upper-body motion and robust lower-body locomotion control, operators can remotely control the humanoid to walk around and explore different environments, while performing diverse manipulation tasks. |
| title | Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Control |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2412.07773 |