Octo: An Open-Source Generalist Robot Policy
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911887957950464 |
|---|---|
| author | Octo Model Team Ghosh, Dibya Walke, Homer Pertsch, Karl Black, Kevin Mees, Oier Dasari, Sudeep Hejna, Joey Kreiman, Tobias Xu, Charles Luo, Jianlan Tan, You Liang Chen, Lawrence Yunliang Sanketi, Pannag Vuong, Quan Xiao, Ted Sadigh, Dorsa Finn, Chelsea Levine, Sergey |
| author_facet | Octo Model Team Ghosh, Dibya Walke, Homer Pertsch, Karl Black, Kevin Mees, Oier Dasari, Sudeep Hejna, Joey Kreiman, Tobias Xu, Charles Luo, Jianlan Tan, You Liang Chen, Lawrence Yunliang Sanketi, Pannag Vuong, Quan Xiao, Ted Sadigh, Dorsa Finn, Chelsea Levine, Sergey |
| contents | Large policies pretrained on diverse robot datasets have the potential to transform robotic learning: instead of training new policies from scratch, such generalist robot policies may be finetuned with only a little in-domain data, yet generalize broadly. However, to be widely applicable across a range of robotic learning scenarios, environments, and tasks, such policies need to handle diverse sensors and action spaces, accommodate a variety of commonly used robotic platforms, and finetune readily and efficiently to new domains. In this work, we aim to lay the groundwork for developing open-source, widely applicable, generalist policies for robotic manipulation. As a first step, we introduce Octo, a large transformer-based policy trained on 800k trajectories from the Open X-Embodiment dataset, the largest robot manipulation dataset to date. It can be instructed via language commands or goal images and can be effectively finetuned to robot setups with new sensory inputs and action spaces within a few hours on standard consumer GPUs. In experiments across 9 robotic platforms, we demonstrate that Octo serves as a versatile policy initialization that can be effectively finetuned to new observation and action spaces. We also perform detailed ablations of design decisions for the Octo model, from architecture to training data, to guide future research on building generalist robot models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_12213 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Octo: An Open-Source Generalist Robot Policy Octo Model Team Ghosh, Dibya Walke, Homer Pertsch, Karl Black, Kevin Mees, Oier Dasari, Sudeep Hejna, Joey Kreiman, Tobias Xu, Charles Luo, Jianlan Tan, You Liang Chen, Lawrence Yunliang Sanketi, Pannag Vuong, Quan Xiao, Ted Sadigh, Dorsa Finn, Chelsea Levine, Sergey Robotics Machine Learning Large policies pretrained on diverse robot datasets have the potential to transform robotic learning: instead of training new policies from scratch, such generalist robot policies may be finetuned with only a little in-domain data, yet generalize broadly. However, to be widely applicable across a range of robotic learning scenarios, environments, and tasks, such policies need to handle diverse sensors and action spaces, accommodate a variety of commonly used robotic platforms, and finetune readily and efficiently to new domains. In this work, we aim to lay the groundwork for developing open-source, widely applicable, generalist policies for robotic manipulation. As a first step, we introduce Octo, a large transformer-based policy trained on 800k trajectories from the Open X-Embodiment dataset, the largest robot manipulation dataset to date. It can be instructed via language commands or goal images and can be effectively finetuned to robot setups with new sensory inputs and action spaces within a few hours on standard consumer GPUs. In experiments across 9 robotic platforms, we demonstrate that Octo serves as a versatile policy initialization that can be effectively finetuned to new observation and action spaces. We also perform detailed ablations of design decisions for the Octo model, from architecture to training data, to guide future research on building generalist robot models. |
| title | Octo: An Open-Source Generalist Robot Policy |
| topic | Robotics Machine Learning |
| url | https://arxiv.org/abs/2405.12213 |