Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916142325432320 |
|---|---|
| author | Yang, Jonathan Glossop, Catherine Bhorkar, Arjun Shah, Dhruv Vuong, Quan Finn, Chelsea Sadigh, Dorsa Levine, Sergey |
| author_facet | Yang, Jonathan Glossop, Catherine Bhorkar, Arjun Shah, Dhruv Vuong, Quan Finn, Chelsea Sadigh, Dorsa Levine, Sergey |
| contents | Recent years in robotics and imitation learning have shown remarkable progress in training large-scale foundation models by leveraging data across a multitude of embodiments. The success of such policies might lead us to wonder: just how diverse can the robots in the training set be while still facilitating positive transfer? In this work, we study this question in the context of heterogeneous embodiments, examining how even seemingly very different domains, such as robotic navigation and manipulation, can provide benefits when included in the training data for the same model. We train a single goal-conditioned policy that is capable of controlling robotic arms, quadcopters, quadrupeds, and mobile bases. We then investigate the extent to which transfer can occur across navigation and manipulation on these embodiments by framing them as a single goal-reaching task. We find that co-training with navigation data can enhance robustness and performance in goal-conditioned manipulation with a wrist-mounted camera. We then deploy our policy trained only from navigation-only and static manipulation-only data on a mobile manipulator, showing that it can control a novel embodiment in a zero-shot manner. These results provide evidence that large-scale robotic policies can benefit from data collected across various embodiments. Further information and robot videos can be found on our project website http://extreme-cross-embodiment.github.io. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_19432 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation Yang, Jonathan Glossop, Catherine Bhorkar, Arjun Shah, Dhruv Vuong, Quan Finn, Chelsea Sadigh, Dorsa Levine, Sergey Robotics 68T40 I.2.9 Recent years in robotics and imitation learning have shown remarkable progress in training large-scale foundation models by leveraging data across a multitude of embodiments. The success of such policies might lead us to wonder: just how diverse can the robots in the training set be while still facilitating positive transfer? In this work, we study this question in the context of heterogeneous embodiments, examining how even seemingly very different domains, such as robotic navigation and manipulation, can provide benefits when included in the training data for the same model. We train a single goal-conditioned policy that is capable of controlling robotic arms, quadcopters, quadrupeds, and mobile bases. We then investigate the extent to which transfer can occur across navigation and manipulation on these embodiments by framing them as a single goal-reaching task. We find that co-training with navigation data can enhance robustness and performance in goal-conditioned manipulation with a wrist-mounted camera. We then deploy our policy trained only from navigation-only and static manipulation-only data on a mobile manipulator, showing that it can control a novel embodiment in a zero-shot manner. These results provide evidence that large-scale robotic policies can benefit from data collected across various embodiments. Further information and robot videos can be found on our project website http://extreme-cross-embodiment.github.io. |
| title | Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation |
| topic | Robotics 68T40 I.2.9 |
| url | https://arxiv.org/abs/2402.19432 |