mindmap: Spatial Memory in Deep Feature Maps for 3D Action Policies
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911195972239360 |
|---|---|
| author | Steiner, Remo Millane, Alexander Tingdahl, David Volk, Clemens Ramasamy, Vikram Yao, Xinjie Du, Peter Pouya, Soha Sheng, Shiwei |
| author_facet | Steiner, Remo Millane, Alexander Tingdahl, David Volk, Clemens Ramasamy, Vikram Yao, Xinjie Du, Peter Pouya, Soha Sheng, Shiwei |
| contents | End-to-end learning of robot control policies, structured as neural networks, has emerged as a promising approach to robotic manipulation. To complete many common tasks, relevant objects are required to pass in and out of a robot's field of view. In these settings, spatial memory - the ability to remember the spatial composition of the scene - is an important competency. However, building such mechanisms into robot learning systems remains an open research problem. We introduce mindmap (Spatial Memory in Deep Feature Maps for 3D Action Policies), a 3D diffusion policy that generates robot trajectories based on a semantic 3D reconstruction of the environment. We show in simulation experiments that our approach is effective at solving tasks where state-of-the-art approaches without memory mechanisms struggle. We release our reconstruction system, training code, and evaluation tasks to spur research in this direction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_20297 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | mindmap: Spatial Memory in Deep Feature Maps for 3D Action Policies Steiner, Remo Millane, Alexander Tingdahl, David Volk, Clemens Ramasamy, Vikram Yao, Xinjie Du, Peter Pouya, Soha Sheng, Shiwei Robotics End-to-end learning of robot control policies, structured as neural networks, has emerged as a promising approach to robotic manipulation. To complete many common tasks, relevant objects are required to pass in and out of a robot's field of view. In these settings, spatial memory - the ability to remember the spatial composition of the scene - is an important competency. However, building such mechanisms into robot learning systems remains an open research problem. We introduce mindmap (Spatial Memory in Deep Feature Maps for 3D Action Policies), a 3D diffusion policy that generates robot trajectories based on a semantic 3D reconstruction of the environment. We show in simulation experiments that our approach is effective at solving tasks where state-of-the-art approaches without memory mechanisms struggle. We release our reconstruction system, training code, and evaluation tasks to spur research in this direction. |
| title | mindmap: Spatial Memory in Deep Feature Maps for 3D Action Policies |
| topic | Robotics |
| url | https://arxiv.org/abs/2509.20297 |