VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908678753353728 |
|---|---|
| author | He, Tairan Wang, Zi Xue, Haoru Ben, Qingwei Luo, Zhengyi Xiao, Wenli Yuan, Ye Da, Xingye Castañeda, Fernando Sastry, Shankar Liu, Changliu Shi, Guanya Fan, Linxi Zhu, Yuke |
| author_facet | He, Tairan Wang, Zi Xue, Haoru Ben, Qingwei Luo, Zhengyi Xiao, Wenli Yuan, Ye Da, Xingye Castañeda, Fernando Sastry, Shankar Liu, Changliu Shi, Guanya Fan, Linxi Zhu, Yuke |
| contents | A key barrier to the real-world deployment of humanoid robots is the lack of autonomous loco-manipulation skills. We introduce VIRAL, a visual sim-to-real framework that learns humanoid loco-manipulation entirely in simulation and deploys it zero-shot to real hardware. VIRAL follows a teacher-student design: a privileged RL teacher, operating on full state, learns long-horizon loco-manipulation using a delta action space and reference state initialization. A vision-based student policy is then distilled from the teacher via large-scale simulation with tiled rendering, trained with a mixture of online DAgger and behavior cloning. We find that compute scale is critical: scaling simulation to tens of GPUs (up to 64) makes both teacher and student training reliable, while low-compute regimes often fail. To bridge the sim-to-real gap, VIRAL combines large-scale visual domain randomization over lighting, materials, camera parameters, image quality, and sensor delays--with real-to-sim alignment of the dexterous hands and cameras. Deployed on a Unitree G1 humanoid, the resulting RGB-based policy performs continuous loco-manipulation for up to 54 cycles, generalizing to diverse spatial and appearance variations without any real-world fine-tuning, and approaching expert-level teleoperation performance. Extensive ablations dissect the key design choices required to make RGB-based humanoid loco-manipulation work in practice. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_15200 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation He, Tairan Wang, Zi Xue, Haoru Ben, Qingwei Luo, Zhengyi Xiao, Wenli Yuan, Ye Da, Xingye Castañeda, Fernando Sastry, Shankar Liu, Changliu Shi, Guanya Fan, Linxi Zhu, Yuke Robotics A key barrier to the real-world deployment of humanoid robots is the lack of autonomous loco-manipulation skills. We introduce VIRAL, a visual sim-to-real framework that learns humanoid loco-manipulation entirely in simulation and deploys it zero-shot to real hardware. VIRAL follows a teacher-student design: a privileged RL teacher, operating on full state, learns long-horizon loco-manipulation using a delta action space and reference state initialization. A vision-based student policy is then distilled from the teacher via large-scale simulation with tiled rendering, trained with a mixture of online DAgger and behavior cloning. We find that compute scale is critical: scaling simulation to tens of GPUs (up to 64) makes both teacher and student training reliable, while low-compute regimes often fail. To bridge the sim-to-real gap, VIRAL combines large-scale visual domain randomization over lighting, materials, camera parameters, image quality, and sensor delays--with real-to-sim alignment of the dexterous hands and cameras. Deployed on a Unitree G1 humanoid, the resulting RGB-based policy performs continuous loco-manipulation for up to 54 cycles, generalizing to diverse spatial and appearance variations without any real-world fine-tuning, and approaching expert-level teleoperation performance. Extensive ablations dissect the key design choices required to make RGB-based humanoid loco-manipulation work in practice. |
| title | VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation |
| topic | Robotics |
| url | https://arxiv.org/abs/2511.15200 |