Visual Imitation Enables Contextual Humanoid Control
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916924444639232 |
|---|---|
| author | Allshire, Arthur Choi, Hongsuk Zhang, Junyi McAllister, David Zhang, Anthony Kim, Chung Min Darrell, Trevor Abbeel, Pieter Malik, Jitendra Kanazawa, Angjoo |
| author_facet | Allshire, Arthur Choi, Hongsuk Zhang, Junyi McAllister, David Zhang, Anthony Kim, Chung Min Darrell, Trevor Abbeel, Pieter Malik, Jitendra Kanazawa, Angjoo |
| contents | How can we teach humanoids to climb staircases and sit on chairs using the surrounding environment context? Arguably, the simplest way is to just show them-casually capture a human motion video and feed it to humanoids. We introduce VIDEOMIMIC, a real-to-sim-to-real pipeline that mines everyday videos, jointly reconstructs the humans and the environment, and produces whole-body control policies for humanoid robots that perform the corresponding skills. We demonstrate the results of our pipeline on real humanoid robots, showing robust, repeatable contextual control such as staircase ascents and descents, sitting and standing from chairs and benches, as well as other dynamic whole-body skills-all from a single policy, conditioned on the environment and global root commands. VIDEOMIMIC offers a scalable path towards teaching humanoids to operate in diverse real-world environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_03729 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Visual Imitation Enables Contextual Humanoid Control Allshire, Arthur Choi, Hongsuk Zhang, Junyi McAllister, David Zhang, Anthony Kim, Chung Min Darrell, Trevor Abbeel, Pieter Malik, Jitendra Kanazawa, Angjoo Robotics Computer Vision and Pattern Recognition How can we teach humanoids to climb staircases and sit on chairs using the surrounding environment context? Arguably, the simplest way is to just show them-casually capture a human motion video and feed it to humanoids. We introduce VIDEOMIMIC, a real-to-sim-to-real pipeline that mines everyday videos, jointly reconstructs the humans and the environment, and produces whole-body control policies for humanoid robots that perform the corresponding skills. We demonstrate the results of our pipeline on real humanoid robots, showing robust, repeatable contextual control such as staircase ascents and descents, sitting and standing from chairs and benches, as well as other dynamic whole-body skills-all from a single policy, conditioned on the environment and global root commands. VIDEOMIMIC offers a scalable path towards teaching humanoids to operate in diverse real-world environments. |
| title | Visual Imitation Enables Contextual Humanoid Control |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.03729 |