EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913948177006592 |
|---|---|
| author | Yang, Ruihan Yu, Qinxi Wu, Yecheng Yan, Rui Li, Borui Cheng, An-Chieh Zou, Xueyan Fang, Yunhao Cheng, Xuxin Qiu, Ri-Zhao Yin, Hongxu Liu, Sifei Han, Song Lu, Yao Wang, Xiaolong |
| author_facet | Yang, Ruihan Yu, Qinxi Wu, Yecheng Yan, Rui Li, Borui Cheng, An-Chieh Zou, Xueyan Fang, Yunhao Cheng, Xuxin Qiu, Ri-Zhao Yin, Hongxu Liu, Sifei Han, Song Lu, Yao Wang, Xiaolong |
| contents | Real robot data collection for imitation learning has led to significant advancements in robotic manipulation. However, the requirement for robot hardware in the process fundamentally constrains the scale of the data. In this paper, we explore training Vision-Language-Action (VLA) models using egocentric human videos. The benefit of using human videos is not only for their scale but more importantly for the richness of scenes and tasks. With a VLA trained on human video that predicts human wrist and hand actions, we can perform Inverse Kinematics and retargeting to convert the human actions to robot actions. We fine-tune the model using a few robot manipulation demonstrations to obtain the robot policy, namely EgoVLA. We propose a simulation benchmark called Ego Humanoid Manipulation Benchmark, where we design diverse bimanual manipulation tasks with demonstrations. We fine-tune and evaluate EgoVLA with Ego Humanoid Manipulation Benchmark and show significant improvements over baselines and ablate the importance of human data. Videos can be found on our website: https://rchalyang.github.io/EgoVLA |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_12440 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos Yang, Ruihan Yu, Qinxi Wu, Yecheng Yan, Rui Li, Borui Cheng, An-Chieh Zou, Xueyan Fang, Yunhao Cheng, Xuxin Qiu, Ri-Zhao Yin, Hongxu Liu, Sifei Han, Song Lu, Yao Wang, Xiaolong Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Real robot data collection for imitation learning has led to significant advancements in robotic manipulation. However, the requirement for robot hardware in the process fundamentally constrains the scale of the data. In this paper, we explore training Vision-Language-Action (VLA) models using egocentric human videos. The benefit of using human videos is not only for their scale but more importantly for the richness of scenes and tasks. With a VLA trained on human video that predicts human wrist and hand actions, we can perform Inverse Kinematics and retargeting to convert the human actions to robot actions. We fine-tune the model using a few robot manipulation demonstrations to obtain the robot policy, namely EgoVLA. We propose a simulation benchmark called Ego Humanoid Manipulation Benchmark, where we design diverse bimanual manipulation tasks with demonstrations. We fine-tune and evaluate EgoVLA with Ego Humanoid Manipulation Benchmark and show significant improvements over baselines and ablate the importance of human data. Videos can be found on our website: https://rchalyang.github.io/EgoVLA |
| title | EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2507.12440 |