HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911725969735680 |
|---|---|
| author | Wang, Zhi He, Botao Yu, Kelin Lee, Seungjae Gao, Ruohan Huang, Furong Aloimonos, Yiannis |
| author_facet | Wang, Zhi He, Botao Yu, Kelin Lee, Seungjae Gao, Ruohan Huang, Furong Aloimonos, Yiannis |
| contents | Human egocentric video captures rich manipulation demonstrations without any robot hardware, yet transferring these skills to robots remains challenging due to the embodiment gap between human and robot in both visual appearance and kinematics. We present HumanEgo, a framework that bridges the embodiment gap by lifting each human demonstration to an entity-level representation of hand-object interaction, and training a flow matching policy with dense auxiliary objectives that amplify supervision from every trajectory. HumanEgo is robot-data-free, hardware-agnostic, data-efficient, and zero-shot human-to-robot transferable. With only 30 minutes of human videos per task, HumanEgo achieves 92.5% average success across four real-world tasks (75% with just 15 minutes), outperforms matched-time robot teleoperation by 41%, and robustly transfers zero-shot across novel robots, cameras, and environments. We release HumanEgo as an easy-to-use, open-source framework for learning robot policies directly from human data: https://github.com/TX-Leo/HumanEgo |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_24934 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos Wang, Zhi He, Botao Yu, Kelin Lee, Seungjae Gao, Ruohan Huang, Furong Aloimonos, Yiannis Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning I.2.9; I.2.6; I.2.10 Human egocentric video captures rich manipulation demonstrations without any robot hardware, yet transferring these skills to robots remains challenging due to the embodiment gap between human and robot in both visual appearance and kinematics. We present HumanEgo, a framework that bridges the embodiment gap by lifting each human demonstration to an entity-level representation of hand-object interaction, and training a flow matching policy with dense auxiliary objectives that amplify supervision from every trajectory. HumanEgo is robot-data-free, hardware-agnostic, data-efficient, and zero-shot human-to-robot transferable. With only 30 minutes of human videos per task, HumanEgo achieves 92.5% average success across four real-world tasks (75% with just 15 minutes), outperforms matched-time robot teleoperation by 41%, and robustly transfers zero-shot across novel robots, cameras, and environments. We release HumanEgo as an easy-to-use, open-source framework for learning robot policies directly from human data: https://github.com/TX-Leo/HumanEgo |
| title | HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning I.2.9; I.2.6; I.2.10 |
| url | https://arxiv.org/abs/2605.24934 |