Motion Before Action: Diffusing Object Motion as Manipulation Condition
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913790296064000 |
|---|---|
| author | Su, Yue Zhan, Xinyu Fang, Hongjie Li, Yong-Lu Lu, Cewu Yang, Lixin |
| author_facet | Su, Yue Zhan, Xinyu Fang, Hongjie Li, Yong-Lu Lu, Cewu Yang, Lixin |
| contents | Inferring object motion representations from observations enhances the performance of robotic manipulation tasks. This paper introduces a new paradigm for robot imitation learning that generates action sequences by reasoning about object motion from visual observations. We propose MBA (Motion Before Action), a novel module that employs two cascaded diffusion processes for object motion generation and robot action generation under object motion guidance. MBA first predicts the future pose sequence of the object based on observations, then uses this sequence as a condition to guide robot action generation. Designed as a plug-and-play component, MBA can be flexibly integrated into existing robotic manipulation policies with diffusion action heads. Extensive experiments in both simulated and real-world environments demonstrate that our approach substantially improves the performance of existing policies across a wide range of manipulation tasks. Project page: https://selen-suyue.github.io/MBApage/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_09658 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Motion Before Action: Diffusing Object Motion as Manipulation Condition Su, Yue Zhan, Xinyu Fang, Hongjie Li, Yong-Lu Lu, Cewu Yang, Lixin Robotics Inferring object motion representations from observations enhances the performance of robotic manipulation tasks. This paper introduces a new paradigm for robot imitation learning that generates action sequences by reasoning about object motion from visual observations. We propose MBA (Motion Before Action), a novel module that employs two cascaded diffusion processes for object motion generation and robot action generation under object motion guidance. MBA first predicts the future pose sequence of the object based on observations, then uses this sequence as a condition to guide robot action generation. Designed as a plug-and-play component, MBA can be flexibly integrated into existing robotic manipulation policies with diffusion action heads. Extensive experiments in both simulated and real-world environments demonstrate that our approach substantially improves the performance of existing policies across a wide range of manipulation tasks. Project page: https://selen-suyue.github.io/MBApage/ |
| title | Motion Before Action: Diffusing Object Motion as Manipulation Condition |
| topic | Robotics |
| url | https://arxiv.org/abs/2411.09658 |