GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914968080744448 |
|---|---|
| author | Cheang, Chi-Lam Chen, Guangzeng Jing, Ya Kong, Tao Li, Hang Li, Yifeng Liu, Yuxiao Wu, Hongtao Xu, Jiafeng Yang, Yichu Zhang, Hanbo Zhu, Minzhao |
| author_facet | Cheang, Chi-Lam Chen, Guangzeng Jing, Ya Kong, Tao Li, Hang Li, Yifeng Liu, Yuxiao Wu, Hongtao Xu, Jiafeng Yang, Yichu Zhang, Hanbo Zhu, Minzhao |
| contents | We present GR-2, a state-of-the-art generalist robot agent for versatile and generalizable robot manipulation. GR-2 is first pre-trained on a vast number of Internet videos to capture the dynamics of the world. This large-scale pre-training, involving 38 million video clips and over 50 billion tokens, equips GR-2 with the ability to generalize across a wide range of robotic tasks and environments during subsequent policy learning. Following this, GR-2 is fine-tuned for both video generation and action prediction using robot trajectories. It exhibits impressive multi-task learning capabilities, achieving an average success rate of 97.7% across more than 100 tasks. Moreover, GR-2 demonstrates exceptional generalization to new, previously unseen scenarios, including novel backgrounds, environments, objects, and tasks. Notably, GR-2 scales effectively with model size, underscoring its potential for continued growth and application. Project page: \url{https://gr2-manipulation.github.io}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06158 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation Cheang, Chi-Lam Chen, Guangzeng Jing, Ya Kong, Tao Li, Hang Li, Yifeng Liu, Yuxiao Wu, Hongtao Xu, Jiafeng Yang, Yichu Zhang, Hanbo Zhu, Minzhao Robotics Computer Vision and Pattern Recognition Machine Learning We present GR-2, a state-of-the-art generalist robot agent for versatile and generalizable robot manipulation. GR-2 is first pre-trained on a vast number of Internet videos to capture the dynamics of the world. This large-scale pre-training, involving 38 million video clips and over 50 billion tokens, equips GR-2 with the ability to generalize across a wide range of robotic tasks and environments during subsequent policy learning. Following this, GR-2 is fine-tuned for both video generation and action prediction using robot trajectories. It exhibits impressive multi-task learning capabilities, achieving an average success rate of 97.7% across more than 100 tasks. Moreover, GR-2 demonstrates exceptional generalization to new, previously unseen scenarios, including novel backgrounds, environments, objects, and tasks. Notably, GR-2 scales effectively with model size, underscoring its potential for continued growth and application. Project page: \url{https://gr2-manipulation.github.io}. |
| title | GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation |
| topic | Robotics Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2410.06158 |