Imagination Policy: Using Generative Point Cloud Models for Learning Manipulation Policies
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912138310713344 |
|---|---|
| author | Huang, Haojie Schmeckpeper, Karl Wang, Dian Biza, Ondrej Qian, Yaoyao Liu, Haotian Jia, Mingxi Platt, Robert Walters, Robin |
| author_facet | Huang, Haojie Schmeckpeper, Karl Wang, Dian Biza, Ondrej Qian, Yaoyao Liu, Haotian Jia, Mingxi Platt, Robert Walters, Robin |
| contents | Humans can imagine goal states during planning and perform actions to match those goals. In this work, we propose Imagination Policy, a novel multi-task key-frame policy network for solving high-precision pick and place tasks. Instead of learning actions directly, Imagination Policy generates point clouds to imagine desired states which are then translated to actions using rigid action estimation. This transforms action inference into a local generative task. We leverage pick and place symmetries underlying the tasks in the generation process and achieve extremely high sample efficiency and generalizability to unseen configurations. Finally, we demonstrate state-of-the-art performance across various tasks on the RLbench benchmark compared with several strong baselines and validate our approach on a real robot. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_11740 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Imagination Policy: Using Generative Point Cloud Models for Learning Manipulation Policies Huang, Haojie Schmeckpeper, Karl Wang, Dian Biza, Ondrej Qian, Yaoyao Liu, Haotian Jia, Mingxi Platt, Robert Walters, Robin Robotics Artificial Intelligence Machine Learning Humans can imagine goal states during planning and perform actions to match those goals. In this work, we propose Imagination Policy, a novel multi-task key-frame policy network for solving high-precision pick and place tasks. Instead of learning actions directly, Imagination Policy generates point clouds to imagine desired states which are then translated to actions using rigid action estimation. This transforms action inference into a local generative task. We leverage pick and place symmetries underlying the tasks in the generation process and achieve extremely high sample efficiency and generalizability to unseen configurations. Finally, we demonstrate state-of-the-art performance across various tasks on the RLbench benchmark compared with several strong baselines and validate our approach on a real robot. |
| title | Imagination Policy: Using Generative Point Cloud Models for Learning Manipulation Policies |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2406.11740 |