Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.23649 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915706477477888 |
|---|---|
| author | Li, Zhe Chi, Cheng Zhu, Boan Wei, Yangyang Bai, Shuanghao Ji, Yuheng Peng, Yibo Huang, Tao Wang, Pengwei Wang, Zhongyuan Chan, S. -H. Gary Xu, Chang Zhang, Shanghang |
| author_facet | Li, Zhe Chi, Cheng Zhu, Boan Wei, Yangyang Bai, Shuanghao Ji, Yuheng Peng, Yibo Huang, Tao Wang, Pengwei Wang, Zhongyuan Chan, S. -H. Gary Xu, Chang Zhang, Shanghang |
| contents | Humans learn locomotion through visual observation, interpreting visual content first before imitating actions. However, state-of-the-art humanoid locomotion systems rely on either curated motion capture trajectories or sparse text commands, leaving a critical gap between visual understanding and control. Text-to-motion methods suffer from semantic sparsity and staged pipeline errors, while video-based approaches only perform mechanical pose mimicry without genuine visual understanding. We propose RoboMirror, the first retargeting-free video-to-locomotion framework embodying "understand before you imitate". Leveraging VLMs, it distills raw egocentric/third-person videos into visual motion intents, which directly condition a diffusion-based policy to generate physically plausible, semantically aligned locomotion without explicit pose reconstruction or retargeting. Extensive experiments validate the effectiveness of RoboMirror, it enables telepresence via egocentric videos, drastically reduces third-person control latency by 80%, and achieves a 3.7% higher task success rate than baselines. By reframing humanoid control around video understanding, we bridge the visual understanding and action gap. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_23649 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RoboMirror: Understand Before You Imitate for Video to Humanoid Locomotion Li, Zhe Chi, Cheng Zhu, Boan Wei, Yangyang Bai, Shuanghao Ji, Yuheng Peng, Yibo Huang, Tao Wang, Pengwei Wang, Zhongyuan Chan, S. -H. Gary Xu, Chang Zhang, Shanghang Robotics Computer Vision and Pattern Recognition Humans learn locomotion through visual observation, interpreting visual content first before imitating actions. However, state-of-the-art humanoid locomotion systems rely on either curated motion capture trajectories or sparse text commands, leaving a critical gap between visual understanding and control. Text-to-motion methods suffer from semantic sparsity and staged pipeline errors, while video-based approaches only perform mechanical pose mimicry without genuine visual understanding. We propose RoboMirror, the first retargeting-free video-to-locomotion framework embodying "understand before you imitate". Leveraging VLMs, it distills raw egocentric/third-person videos into visual motion intents, which directly condition a diffusion-based policy to generate physically plausible, semantically aligned locomotion without explicit pose reconstruction or retargeting. Extensive experiments validate the effectiveness of RoboMirror, it enables telepresence via egocentric videos, drastically reduces third-person control latency by 80%, and achieves a 3.7% higher task success rate than baselines. By reframing humanoid control around video understanding, we bridge the visual understanding and action gap. |
| title | RoboMirror: Understand Before You Imitate for Video to Humanoid Locomotion |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.23649 |