Ego to World: Collaborative Spatial Reasoning in Embodied Systems via Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915865279070208 |
|---|---|
| author | Zhou, Heng Kang, Li Qin, Yiran Song, Xiufeng Yu, Ao Zhang, Zilu Song, Haoming Xu, Kaixin Fan, Yuchen Zhou, Dongzhan Liu, Xiaohong Zhang, Ruimao Torr, Philip Bai, Lei Yin, Zhenfei |
| author_facet | Zhou, Heng Kang, Li Qin, Yiran Song, Xiufeng Yu, Ao Zhang, Zilu Song, Haoming Xu, Kaixin Fan, Yuchen Zhou, Dongzhan Liu, Xiaohong Zhang, Ruimao Torr, Philip Bai, Lei Yin, Zhenfei |
| contents | Understanding the world from distributed, partial viewpoints is a fundamental challenge for embodied multi-agent systems. Each agent perceives the environment through an ego-centric view that is often limited by occlusion and ambiguity. To study this problem, we introduce the Ego-to-World (E2W) benchmark, which evaluates a vision-language model's ability to fuse heterogeneous viewpoints across three tasks: (i) global counting, (ii) relational location reasoning, and (iii) action-oriented grasping that requires predicting view-specific image coordinates. To address this setting, we propose CoRL, a two-stage framework that combines Chain-of-Thought supervised fine-tuning with reinforcement learning using Group-Relative Policy Optimization. Its core component, the Cross-View Spatial Reward (CVSR), provides dense task-aligned feedback by linking reasoning steps to visual evidence, ensuring coherent cross-view entity resolution, and guiding the model toward correct final predictions. Experiments on E2W show that CoRL consistently surpasses strong proprietary and open-source baselines on both reasoning and perception-grounding metrics, while ablations further confirm the necessity of each CVSR component. Beyond that, CoRL generalizes to external spatial reasoning benchmarks and enables effective real-world multi-robot manipulation with calibrated multi-camera rigs, demonstrating cross-view localization and successful grasp-and-place execution. Together, E2W and CoRL provide a principled foundation for learning world-centric scene understanding from distributed, ego-centric observations, advancing collaborative embodied AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_14811 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Ego to World: Collaborative Spatial Reasoning in Embodied Systems via Reinforcement Learning Zhou, Heng Kang, Li Qin, Yiran Song, Xiufeng Yu, Ao Zhang, Zilu Song, Haoming Xu, Kaixin Fan, Yuchen Zhou, Dongzhan Liu, Xiaohong Zhang, Ruimao Torr, Philip Bai, Lei Yin, Zhenfei Robotics Computer Vision and Pattern Recognition Understanding the world from distributed, partial viewpoints is a fundamental challenge for embodied multi-agent systems. Each agent perceives the environment through an ego-centric view that is often limited by occlusion and ambiguity. To study this problem, we introduce the Ego-to-World (E2W) benchmark, which evaluates a vision-language model's ability to fuse heterogeneous viewpoints across three tasks: (i) global counting, (ii) relational location reasoning, and (iii) action-oriented grasping that requires predicting view-specific image coordinates. To address this setting, we propose CoRL, a two-stage framework that combines Chain-of-Thought supervised fine-tuning with reinforcement learning using Group-Relative Policy Optimization. Its core component, the Cross-View Spatial Reward (CVSR), provides dense task-aligned feedback by linking reasoning steps to visual evidence, ensuring coherent cross-view entity resolution, and guiding the model toward correct final predictions. Experiments on E2W show that CoRL consistently surpasses strong proprietary and open-source baselines on both reasoning and perception-grounding metrics, while ablations further confirm the necessity of each CVSR component. Beyond that, CoRL generalizes to external spatial reasoning benchmarks and enables effective real-world multi-robot manipulation with calibrated multi-camera rigs, demonstrating cross-view localization and successful grasp-and-place execution. Together, E2W and CoRL provide a principled foundation for learning world-centric scene understanding from distributed, ego-centric observations, advancing collaborative embodied AI. |
| title | Ego to World: Collaborative Spatial Reasoning in Embodied Systems via Reinforcement Learning |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.14811 |