Actron3D: Learning Actionable Neural Functions from Videos for Transferable Robotic Manipulation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909847028498432 |
|---|---|
| author | Zhang, Anran Chen, Hanzhi Burkhardt, Yannick Zhong, Yao Betz, Johannes Oleynikova, Helen Leutenegger, Stefan |
| author_facet | Zhang, Anran Chen, Hanzhi Burkhardt, Yannick Zhong, Yao Betz, Johannes Oleynikova, Helen Leutenegger, Stefan |
| contents | We present Actron3D, a framework that enables robots to acquire transferable 6-DoF manipulation skills from just a few monocular, uncalibrated, RGB-only human videos. At its core lies the Neural Affordance Function, a compact object-centric representation that distills actionable cues from diverse uncalibrated videos-geometry, visual appearance, and affordance-into a lightweight neural network, forming a memory bank of manipulation skills. During deployment, we adopt a pipeline that retrieves relevant affordance functions and transfers precise 6-DoF manipulation policies via coarse-to-fine optimization, enabled by continuous queries to the multimodal features encoded in the neural functions. Experiments in both simulation and the real world demonstrate that Actron3D significantly outperforms prior methods, achieving a 14.9 percentage point improvement in average success rate across 13 tasks while requiring only 2-3 demonstration videos per task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_12971 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Actron3D: Learning Actionable Neural Functions from Videos for Transferable Robotic Manipulation Zhang, Anran Chen, Hanzhi Burkhardt, Yannick Zhong, Yao Betz, Johannes Oleynikova, Helen Leutenegger, Stefan Robotics We present Actron3D, a framework that enables robots to acquire transferable 6-DoF manipulation skills from just a few monocular, uncalibrated, RGB-only human videos. At its core lies the Neural Affordance Function, a compact object-centric representation that distills actionable cues from diverse uncalibrated videos-geometry, visual appearance, and affordance-into a lightweight neural network, forming a memory bank of manipulation skills. During deployment, we adopt a pipeline that retrieves relevant affordance functions and transfers precise 6-DoF manipulation policies via coarse-to-fine optimization, enabled by continuous queries to the multimodal features encoded in the neural functions. Experiments in both simulation and the real world demonstrate that Actron3D significantly outperforms prior methods, achieving a 14.9 percentage point improvement in average success rate across 13 tasks while requiring only 2-3 demonstration videos per task. |
| title | Actron3D: Learning Actionable Neural Functions from Videos for Transferable Robotic Manipulation |
| topic | Robotics |
| url | https://arxiv.org/abs/2510.12971 |