Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.19709 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915885620396032 |
|---|---|
| author | Xu, Weisheng Li, Jian Gu, Yi Yang, Bin Chen, Haodong Lin, Shuyi Zhou, Mingqian Tan, Jing Wu, Qiwei Jiang, Xiangrui Wang, Taowen Wen, Jiawen Liang, Qiwei Zhang, Jiaxi Xu, Renjing |
| author_facet | Xu, Weisheng Li, Jian Gu, Yi Yang, Bin Chen, Haodong Lin, Shuyi Zhou, Mingqian Tan, Jing Wu, Qiwei Jiang, Xiangrui Wang, Taowen Wen, Jiawen Liang, Qiwei Zhang, Jiaxi Xu, Renjing |
| contents | Equipping humanoid robots with versatile interaction skills typically requires either extensive policy training or explicit human-to-robot motion retargeting. However, learning-based policies face prohibitive data collection costs. Meanwhile, retargeting relies on human-centric pose estimation (e.g., SMPL), introducing a morphology gap. Skeletal scale mismatches result in severe spatial misalignments when mapped to robots, compromising interaction success. In this work, we propose Dream2Act, a robot-centric framework enabling zero-shot interaction through generative video synthesis. Given a third-person image of the robot and target object, our framework leverages video generation models to envision the robot completing the task with morphology-consistent motion. We employ a high-fidelity pose extraction system to recover physically feasible, robot-native joint trajectories from these synthesized dreams, subsequently executed via a general-purpose whole-body controller. Operating strictly within the robot-native coordinate space, Dream2Act avoids retargeting errors and eliminates task-specific policy training. We evaluate Dream2Act on the Unitree G1 across four whole-body mobile interaction tasks: ball kicking, sofa sitting, bag punching, and box hugging. Dream2Act achieves a 37.5% overall success rate, compared to 0% for conventional retargeting. While retargeting fails to establish correct physical contacts due to the morphology gap (with errors compounded during locomotion), Dream2Act maintains robot-consistent spatial alignment, enabling reliable contact formation and substantially higher task completion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_19709 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Morphology-Consistent Humanoid Interaction through Robot-Centric Video Synthesis Xu, Weisheng Li, Jian Gu, Yi Yang, Bin Chen, Haodong Lin, Shuyi Zhou, Mingqian Tan, Jing Wu, Qiwei Jiang, Xiangrui Wang, Taowen Wen, Jiawen Liang, Qiwei Zhang, Jiaxi Xu, Renjing Robotics Equipping humanoid robots with versatile interaction skills typically requires either extensive policy training or explicit human-to-robot motion retargeting. However, learning-based policies face prohibitive data collection costs. Meanwhile, retargeting relies on human-centric pose estimation (e.g., SMPL), introducing a morphology gap. Skeletal scale mismatches result in severe spatial misalignments when mapped to robots, compromising interaction success. In this work, we propose Dream2Act, a robot-centric framework enabling zero-shot interaction through generative video synthesis. Given a third-person image of the robot and target object, our framework leverages video generation models to envision the robot completing the task with morphology-consistent motion. We employ a high-fidelity pose extraction system to recover physically feasible, robot-native joint trajectories from these synthesized dreams, subsequently executed via a general-purpose whole-body controller. Operating strictly within the robot-native coordinate space, Dream2Act avoids retargeting errors and eliminates task-specific policy training. We evaluate Dream2Act on the Unitree G1 across four whole-body mobile interaction tasks: ball kicking, sofa sitting, bag punching, and box hugging. Dream2Act achieves a 37.5% overall success rate, compared to 0% for conventional retargeting. While retargeting fails to establish correct physical contacts due to the morphology gap (with errors compounded during locomotion), Dream2Act maintains robot-consistent spatial alignment, enabling reliable contact formation and substantially higher task completion. |
| title | Morphology-Consistent Humanoid Interaction through Robot-Centric Video Synthesis |
| topic | Robotics |
| url | https://arxiv.org/abs/2603.19709 |