Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910951382450176 |
|---|---|
| author | Luo, Zhengyi Tessler, Chen Lin, Toru Yuan, Ye He, Tairan Xiao, Wenli Guo, Yunrong Chechik, Gal Kitani, Kris Fan, Linxi Zhu, Yuke |
| author_facet | Luo, Zhengyi Tessler, Chen Lin, Toru Yuan, Ye He, Tairan Xiao, Wenli Guo, Yunrong Chechik, Gal Kitani, Kris Fan, Linxi Zhu, Yuke |
| contents | Human behavior is fundamentally shaped by visual perception -- our ability to interact with the world depends on actively gathering relevant information and adapting our movements accordingly. Behaviors like searching for objects, reaching, and hand-eye coordination naturally emerge from the structure of our sensory system. Inspired by these principles, we introduce Perceptive Dexterous Control (PDC), a framework for vision-driven dexterous whole-body control with simulated humanoids. PDC operates solely on egocentric vision for task specification, enabling object search, target placement, and skill selection through visual cues, without relying on privileged state information (e.g., 3D object positions and geometries). This perception-as-interface paradigm enables learning a single policy to perform multiple household tasks, including reaching, grasping, placing, and articulated object manipulation. We also show that training from scratch with reinforcement learning can produce emergent behaviors such as active search. These results demonstrate how vision-driven control and complex tasks induce human-like behaviors and can serve as the key ingredients in closing the perception-action loop for animation, robotics, and embodied AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_12278 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learning Luo, Zhengyi Tessler, Chen Lin, Toru Yuan, Ye He, Tairan Xiao, Wenli Guo, Yunrong Chechik, Gal Kitani, Kris Fan, Linxi Zhu, Yuke Robotics Computer Vision and Pattern Recognition Human behavior is fundamentally shaped by visual perception -- our ability to interact with the world depends on actively gathering relevant information and adapting our movements accordingly. Behaviors like searching for objects, reaching, and hand-eye coordination naturally emerge from the structure of our sensory system. Inspired by these principles, we introduce Perceptive Dexterous Control (PDC), a framework for vision-driven dexterous whole-body control with simulated humanoids. PDC operates solely on egocentric vision for task specification, enabling object search, target placement, and skill selection through visual cues, without relying on privileged state information (e.g., 3D object positions and geometries). This perception-as-interface paradigm enables learning a single policy to perform multiple household tasks, including reaching, grasping, placing, and articulated object manipulation. We also show that training from scratch with reinforcement learning can produce emergent behaviors such as active search. These results demonstrate how vision-driven control and complex tasks induce human-like behaviors and can serve as the key ingredients in closing the perception-action loop for animation, robotics, and embodied AI. |
| title | Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learning |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.12278 |