Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.20020 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916662739992576 |
|---|---|
| author | Gemini Robotics Team Abeyruwan, Saminda Ainslie, Joshua Alayrac, Jean-Baptiste Arenas, Montserrat Gonzalez Armstrong, Travis Balakrishna, Ashwin Baruch, Robert Bauza, Maria Blokzijl, Michiel Bohez, Steven Bousmalis, Konstantinos Brohan, Anthony Buschmann, Thomas Byravan, Arunkumar Cabi, Serkan Caluwaerts, Ken Casarini, Federico Chang, Oscar Chen, Jose Enrique Chen, Xi Chiang, Hao-Tien Lewis Choromanski, Krzysztof D'Ambrosio, David Dasari, Sudeep Davchev, Todor Devin, Coline Di Palo, Norman Ding, Tianli Dostmohamed, Adil Driess, Danny Du, Yilun Dwibedi, Debidatta Elabd, Michael Fantacci, Claudio Fong, Cody Frey, Erik Fu, Chuyuan Giustina, Marissa Gopalakrishnan, Keerthana Graesser, Laura Hasenclever, Leonard Heess, Nicolas Hernaez, Brandon Herzog, Alexander Hofer, R. Alex Humplik, Jan Iscen, Atil Jacob, Mithun George Jain, Deepali Julian, Ryan Kalashnikov, Dmitry Karagozler, M. Emre Karp, Stefani Kew, Chase Kirkland, Jerad Kirmani, Sean Kuang, Yuheng Lampe, Thomas Laurens, Antoine Leal, Isabel Lee, Alex X. Lee, Tsang-Wei Edward Liang, Jacky Lin, Yixin Maddineni, Sharath Majumdar, Anirudha Michaely, Assaf Hurwitz Moreno, Robert Neunert, Michael Nori, Francesco Parada, Carolina Parisotto, Emilio Pastor, Peter Pooley, Acorn Rao, Kanishka Reymann, Krista Sadigh, Dorsa Saliceti, Stefano Sanketi, Pannag Sermanet, Pierre Shah, Dhruv Sharma, Mohit Shea, Kathryn Shu, Charles Sindhwani, Vikas Singh, Sumeet Soricut, Radu Springenberg, Jost Tobias Sterneck, Rachel Surdulescu, Razvan Tan, Jie Tompson, Jonathan Vanhoucke, Vincent Varley, Jake Vesom, Grace Vezzani, Giulia Vinyals, Oriol Wahid, Ayzaan Welker, Stefan Wohlhart, Paul Xia, Fei Xiao, Ted Xie, Annie Xie, Jinyu Xu, Peng Xu, Sichun Xu, Ying Xu, Zhuo Yang, Yuxiang Yao, Rui Yaroshenko, Sergey Yu, Wenhao Yuan, Wentao Zhang, Jingwei Zhang, Tingnan Zhou, Allan Zhou, Yuxiang |
| author_facet | Gemini Robotics Team Abeyruwan, Saminda Ainslie, Joshua Alayrac, Jean-Baptiste Arenas, Montserrat Gonzalez Armstrong, Travis Balakrishna, Ashwin Baruch, Robert Bauza, Maria Blokzijl, Michiel Bohez, Steven Bousmalis, Konstantinos Brohan, Anthony Buschmann, Thomas Byravan, Arunkumar Cabi, Serkan Caluwaerts, Ken Casarini, Federico Chang, Oscar Chen, Jose Enrique Chen, Xi Chiang, Hao-Tien Lewis Choromanski, Krzysztof D'Ambrosio, David Dasari, Sudeep Davchev, Todor Devin, Coline Di Palo, Norman Ding, Tianli Dostmohamed, Adil Driess, Danny Du, Yilun Dwibedi, Debidatta Elabd, Michael Fantacci, Claudio Fong, Cody Frey, Erik Fu, Chuyuan Giustina, Marissa Gopalakrishnan, Keerthana Graesser, Laura Hasenclever, Leonard Heess, Nicolas Hernaez, Brandon Herzog, Alexander Hofer, R. Alex Humplik, Jan Iscen, Atil Jacob, Mithun George Jain, Deepali Julian, Ryan Kalashnikov, Dmitry Karagozler, M. Emre Karp, Stefani Kew, Chase Kirkland, Jerad Kirmani, Sean Kuang, Yuheng Lampe, Thomas Laurens, Antoine Leal, Isabel Lee, Alex X. Lee, Tsang-Wei Edward Liang, Jacky Lin, Yixin Maddineni, Sharath Majumdar, Anirudha Michaely, Assaf Hurwitz Moreno, Robert Neunert, Michael Nori, Francesco Parada, Carolina Parisotto, Emilio Pastor, Peter Pooley, Acorn Rao, Kanishka Reymann, Krista Sadigh, Dorsa Saliceti, Stefano Sanketi, Pannag Sermanet, Pierre Shah, Dhruv Sharma, Mohit Shea, Kathryn Shu, Charles Sindhwani, Vikas Singh, Sumeet Soricut, Radu Springenberg, Jost Tobias Sterneck, Rachel Surdulescu, Razvan Tan, Jie Tompson, Jonathan Vanhoucke, Vincent Varley, Jake Vesom, Grace Vezzani, Giulia Vinyals, Oriol Wahid, Ayzaan Welker, Stefan Wohlhart, Paul Xia, Fei Xiao, Ted Xie, Annie Xie, Jinyu Xu, Peng Xu, Sichun Xu, Ying Xu, Zhuo Yang, Yuxiang Yao, Rui Yaroshenko, Sergey Yu, Wenhao Yuan, Wentao Zhang, Jingwei Zhang, Tingnan Zhou, Allan Zhou, Yuxiang |
| contents | Recent advancements in large multimodal models have led to the emergence of remarkable generalist capabilities in digital domains, yet their translation to physical agents such as robots remains a significant challenge. This report introduces a new family of AI models purposefully designed for robotics and built upon the foundation of Gemini 2.0. We present Gemini Robotics, an advanced Vision-Language-Action (VLA) generalist model capable of directly controlling robots. Gemini Robotics executes smooth and reactive movements to tackle a wide range of complex manipulation tasks while also being robust to variations in object types and positions, handling unseen environments as well as following diverse, open vocabulary instructions. We show that with additional fine-tuning, Gemini Robotics can be specialized to new capabilities including solving long-horizon, highly dexterous tasks, learning new short-horizon tasks from as few as 100 demonstrations and adapting to completely novel robot embodiments. This is made possible because Gemini Robotics builds on top of the Gemini Robotics-ER model, the second model we introduce in this work. Gemini Robotics-ER (Embodied Reasoning) extends Gemini's multimodal reasoning capabilities into the physical world, with enhanced spatial and temporal understanding. This enables capabilities relevant to robotics including object detection, pointing, trajectory and grasp prediction, as well as multi-view correspondence and 3D bounding box predictions. We show how this novel combination can support a variety of robotics applications. We also discuss and address important safety considerations related to this new class of robotics foundation models. The Gemini Robotics family marks a substantial step towards developing general-purpose robots that realizes AI's potential in the physical world. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_20020 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Gemini Robotics: Bringing AI into the Physical World Gemini Robotics Team Abeyruwan, Saminda Ainslie, Joshua Alayrac, Jean-Baptiste Arenas, Montserrat Gonzalez Armstrong, Travis Balakrishna, Ashwin Baruch, Robert Bauza, Maria Blokzijl, Michiel Bohez, Steven Bousmalis, Konstantinos Brohan, Anthony Buschmann, Thomas Byravan, Arunkumar Cabi, Serkan Caluwaerts, Ken Casarini, Federico Chang, Oscar Chen, Jose Enrique Chen, Xi Chiang, Hao-Tien Lewis Choromanski, Krzysztof D'Ambrosio, David Dasari, Sudeep Davchev, Todor Devin, Coline Di Palo, Norman Ding, Tianli Dostmohamed, Adil Driess, Danny Du, Yilun Dwibedi, Debidatta Elabd, Michael Fantacci, Claudio Fong, Cody Frey, Erik Fu, Chuyuan Giustina, Marissa Gopalakrishnan, Keerthana Graesser, Laura Hasenclever, Leonard Heess, Nicolas Hernaez, Brandon Herzog, Alexander Hofer, R. Alex Humplik, Jan Iscen, Atil Jacob, Mithun George Jain, Deepali Julian, Ryan Kalashnikov, Dmitry Karagozler, M. Emre Karp, Stefani Kew, Chase Kirkland, Jerad Kirmani, Sean Kuang, Yuheng Lampe, Thomas Laurens, Antoine Leal, Isabel Lee, Alex X. Lee, Tsang-Wei Edward Liang, Jacky Lin, Yixin Maddineni, Sharath Majumdar, Anirudha Michaely, Assaf Hurwitz Moreno, Robert Neunert, Michael Nori, Francesco Parada, Carolina Parisotto, Emilio Pastor, Peter Pooley, Acorn Rao, Kanishka Reymann, Krista Sadigh, Dorsa Saliceti, Stefano Sanketi, Pannag Sermanet, Pierre Shah, Dhruv Sharma, Mohit Shea, Kathryn Shu, Charles Sindhwani, Vikas Singh, Sumeet Soricut, Radu Springenberg, Jost Tobias Sterneck, Rachel Surdulescu, Razvan Tan, Jie Tompson, Jonathan Vanhoucke, Vincent Varley, Jake Vesom, Grace Vezzani, Giulia Vinyals, Oriol Wahid, Ayzaan Welker, Stefan Wohlhart, Paul Xia, Fei Xiao, Ted Xie, Annie Xie, Jinyu Xu, Peng Xu, Sichun Xu, Ying Xu, Zhuo Yang, Yuxiang Yao, Rui Yaroshenko, Sergey Yu, Wenhao Yuan, Wentao Zhang, Jingwei Zhang, Tingnan Zhou, Allan Zhou, Yuxiang Robotics Recent advancements in large multimodal models have led to the emergence of remarkable generalist capabilities in digital domains, yet their translation to physical agents such as robots remains a significant challenge. This report introduces a new family of AI models purposefully designed for robotics and built upon the foundation of Gemini 2.0. We present Gemini Robotics, an advanced Vision-Language-Action (VLA) generalist model capable of directly controlling robots. Gemini Robotics executes smooth and reactive movements to tackle a wide range of complex manipulation tasks while also being robust to variations in object types and positions, handling unseen environments as well as following diverse, open vocabulary instructions. We show that with additional fine-tuning, Gemini Robotics can be specialized to new capabilities including solving long-horizon, highly dexterous tasks, learning new short-horizon tasks from as few as 100 demonstrations and adapting to completely novel robot embodiments. This is made possible because Gemini Robotics builds on top of the Gemini Robotics-ER model, the second model we introduce in this work. Gemini Robotics-ER (Embodied Reasoning) extends Gemini's multimodal reasoning capabilities into the physical world, with enhanced spatial and temporal understanding. This enables capabilities relevant to robotics including object detection, pointing, trajectory and grasp prediction, as well as multi-view correspondence and 3D bounding box predictions. We show how this novel combination can support a variety of robotics applications. We also discuss and address important safety considerations related to this new class of robotics foundation models. The Gemini Robotics family marks a substantial step towards developing general-purpose robots that realizes AI's potential in the physical world. |
| title | Gemini Robotics: Bringing AI into the Physical World |
| topic | Robotics |
| url | https://arxiv.org/abs/2503.20020 |