SEM: Enhancing Spatial Understanding for Robust Robot Manipulation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908556242976768 |
|---|---|
| author | Lin, Xuewu Lin, Tianwei Huang, Lichao Xie, Hongyu Jin, Yiwei Li, Keyu Su, Zhizhong |
| author_facet | Lin, Xuewu Lin, Tianwei Huang, Lichao Xie, Hongyu Jin, Yiwei Li, Keyu Su, Zhizhong |
| contents | A key challenge in robot manipulation lies in developing policy models with strong spatial understanding, the ability to reason about 3D geometry, object relations, and robot embodiment. Existing methods often fall short: 3D point cloud models lack semantic abstraction, while 2D image encoders struggle with spatial reasoning. To address this, we propose SEM (Spatial Enhanced Manipulation model), a novel diffusion-based policy framework that explicitly enhances spatial understanding from two complementary perspectives. A spatial enhancer augments visual representations with 3D geometric context, while a robot state encoder captures embodiment-aware structure through graphbased modeling of joint dependencies. By integrating these modules, SEM significantly improves spatial understanding, leading to robust and generalizable manipulation across diverse tasks that outperform existing baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_16196 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SEM: Enhancing Spatial Understanding for Robust Robot Manipulation Lin, Xuewu Lin, Tianwei Huang, Lichao Xie, Hongyu Jin, Yiwei Li, Keyu Su, Zhizhong Robotics Artificial Intelligence Computer Vision and Pattern Recognition A key challenge in robot manipulation lies in developing policy models with strong spatial understanding, the ability to reason about 3D geometry, object relations, and robot embodiment. Existing methods often fall short: 3D point cloud models lack semantic abstraction, while 2D image encoders struggle with spatial reasoning. To address this, we propose SEM (Spatial Enhanced Manipulation model), a novel diffusion-based policy framework that explicitly enhances spatial understanding from two complementary perspectives. A spatial enhancer augments visual representations with 3D geometric context, while a robot state encoder captures embodiment-aware structure through graphbased modeling of joint dependencies. By integrating these modules, SEM significantly improves spatial understanding, leading to robust and generalizable manipulation across diverse tasks that outperform existing baselines. |
| title | SEM: Enhancing Spatial Understanding for Robust Robot Manipulation |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.16196 |