Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913930361700352 |
|---|---|
| author | Niu, Yaru Zhang, Yunzhe Yu, Mingyang Lin, Changyi Li, Chenhao Wang, Yikai Yang, Yuxiang Yu, Wenhao Zhang, Tingnan Li, Zhenzhen Francis, Jonathan Chen, Bingqing Tan, Jie Zhao, Ding |
| author_facet | Niu, Yaru Zhang, Yunzhe Yu, Mingyang Lin, Changyi Li, Chenhao Wang, Yikai Yang, Yuxiang Yu, Wenhao Zhang, Tingnan Li, Zhenzhen Francis, Jonathan Chen, Bingqing Tan, Jie Zhao, Ding |
| contents | Quadrupedal robots have demonstrated impressive locomotion capabilities in complex environments, but equipping them with autonomous versatile manipulation skills in a scalable way remains a significant challenge. In this work, we introduce a cross-embodiment imitation learning system for quadrupedal manipulation, leveraging data collected from both humans and LocoMan, a quadruped equipped with multiple manipulation modes. Specifically, we develop a teleoperation and data collection pipeline, which unifies and modularizes the observation and action spaces of the human and the robot. To effectively leverage the collected data, we propose an efficient modularized architecture that supports co-training and pretraining on structured modality-aligned data across different embodiments. Additionally, we construct the first manipulation dataset for the LocoMan robot, covering various household tasks in both unimanual and bimanual modes, supplemented by a corresponding human dataset. We validate our system on six real-world manipulation tasks, where it achieves an average success rate improvement of 41.9% overall and 79.7% under out-of-distribution (OOD) settings compared to the baseline. Pretraining with human data contributes a 38.6% success rate improvement overall and 82.7% under OOD settings, enabling consistently better performance with only half the amount of robot data. Our code, hardware, and data are open-sourced at: https://human2bots.github.io. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_16475 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining Niu, Yaru Zhang, Yunzhe Yu, Mingyang Lin, Changyi Li, Chenhao Wang, Yikai Yang, Yuxiang Yu, Wenhao Zhang, Tingnan Li, Zhenzhen Francis, Jonathan Chen, Bingqing Tan, Jie Zhao, Ding Robotics Artificial Intelligence Machine Learning Quadrupedal robots have demonstrated impressive locomotion capabilities in complex environments, but equipping them with autonomous versatile manipulation skills in a scalable way remains a significant challenge. In this work, we introduce a cross-embodiment imitation learning system for quadrupedal manipulation, leveraging data collected from both humans and LocoMan, a quadruped equipped with multiple manipulation modes. Specifically, we develop a teleoperation and data collection pipeline, which unifies and modularizes the observation and action spaces of the human and the robot. To effectively leverage the collected data, we propose an efficient modularized architecture that supports co-training and pretraining on structured modality-aligned data across different embodiments. Additionally, we construct the first manipulation dataset for the LocoMan robot, covering various household tasks in both unimanual and bimanual modes, supplemented by a corresponding human dataset. We validate our system on six real-world manipulation tasks, where it achieves an average success rate improvement of 41.9% overall and 79.7% under out-of-distribution (OOD) settings compared to the baseline. Pretraining with human data contributes a 38.6% success rate improvement overall and 82.7% under OOD settings, enabling consistently better performance with only half the amount of robot data. Our code, hardware, and data are open-sourced at: https://human2bots.github.io. |
| title | Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2506.16475 |