Towards Proprioception-Aware Embodied Planning for Dual-Arm Humanoid Robots
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918161126785024 |
|---|---|
| author | Li, Boyu He, Siyuan Xu, Hang Yuan, Haoqi Xu, Xinrun Zang, Yu Hu, Liwei Yue, Junpeng Jiang, Zhenxiong Hu, Pengbo Karlsson, Börje F. Tang, Yehui Lu, Zongqing |
| author_facet | Li, Boyu He, Siyuan Xu, Hang Yuan, Haoqi Xu, Xinrun Zang, Yu Hu, Liwei Yue, Junpeng Jiang, Zhenxiong Hu, Pengbo Karlsson, Börje F. Tang, Yehui Lu, Zongqing |
| contents | In recent years, Multimodal Large Language Models (MLLMs) have demonstrated the ability to serve as high-level planners, enabling robots to follow complex human instructions. However, their effectiveness, especially in long-horizon tasks involving dual-arm humanoid robots, remains limited. This limitation arises from two main challenges: (i) the absence of simulation platforms that systematically support task evaluation and data collection for humanoid robots, and (ii) the insufficient embodiment awareness of current MLLMs, which hinders reasoning about dual-arm selection logic and body positions during planning. To address these issues, we present DualTHOR, a new dual-arm humanoid simulator, with continuous transition and a contingency mechanism. Building on this platform, we propose Proprio-MLLM, a model that enhances embodiment awareness by incorporating proprioceptive information with motion-based position embedding and a cross-spatial encoder. Experiments show that, while existing MLLMs struggle in this environment, Proprio-MLLM achieves an average improvement of 19.75% in planning performance. Our work provides both an essential simulation platform and an effective model to advance embodied intelligence in humanoid robotics. The code is available at https://anonymous.4open.science/r/DualTHOR-5F3B. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_07882 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Proprioception-Aware Embodied Planning for Dual-Arm Humanoid Robots Li, Boyu He, Siyuan Xu, Hang Yuan, Haoqi Xu, Xinrun Zang, Yu Hu, Liwei Yue, Junpeng Jiang, Zhenxiong Hu, Pengbo Karlsson, Börje F. Tang, Yehui Lu, Zongqing Robotics In recent years, Multimodal Large Language Models (MLLMs) have demonstrated the ability to serve as high-level planners, enabling robots to follow complex human instructions. However, their effectiveness, especially in long-horizon tasks involving dual-arm humanoid robots, remains limited. This limitation arises from two main challenges: (i) the absence of simulation platforms that systematically support task evaluation and data collection for humanoid robots, and (ii) the insufficient embodiment awareness of current MLLMs, which hinders reasoning about dual-arm selection logic and body positions during planning. To address these issues, we present DualTHOR, a new dual-arm humanoid simulator, with continuous transition and a contingency mechanism. Building on this platform, we propose Proprio-MLLM, a model that enhances embodiment awareness by incorporating proprioceptive information with motion-based position embedding and a cross-spatial encoder. Experiments show that, while existing MLLMs struggle in this environment, Proprio-MLLM achieves an average improvement of 19.75% in planning performance. Our work provides both an essential simulation platform and an effective model to advance embodied intelligence in humanoid robotics. The code is available at https://anonymous.4open.science/r/DualTHOR-5F3B. |
| title | Towards Proprioception-Aware Embodied Planning for Dual-Arm Humanoid Robots |
| topic | Robotics |
| url | https://arxiv.org/abs/2510.07882 |