GRUtopia: Dream General Robots in a City at Scale
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929421461487616 |
|---|---|
| author | Wang, Hanqing Chen, Jiahe Huang, Wensi Ben, Qingwei Wang, Tai Mi, Boyu Huang, Tao Zhao, Siheng Chen, Yilun Yang, Sizhe Cao, Peizhou Yu, Wenye Ye, Zichao Li, Jialun Long, Junfeng Wang, Zirui Wang, Huiling Zhao, Ying Tu, Zhongying Qiao, Yu Lin, Dahua Pang, Jiangmiao |
| author_facet | Wang, Hanqing Chen, Jiahe Huang, Wensi Ben, Qingwei Wang, Tai Mi, Boyu Huang, Tao Zhao, Siheng Chen, Yilun Yang, Sizhe Cao, Peizhou Yu, Wenye Ye, Zichao Li, Jialun Long, Junfeng Wang, Zirui Wang, Huiling Zhao, Ying Tu, Zhongying Qiao, Yu Lin, Dahua Pang, Jiangmiao |
| contents | Recent works have been exploring the scaling laws in the field of Embodied AI. Given the prohibitive costs of collecting real-world data, we believe the Simulation-to-Real (Sim2Real) paradigm is a crucial step for scaling the learning of embodied models. This paper introduces project GRUtopia, the first simulated interactive 3D society designed for various robots. It features several advancements: (a) The scene dataset, GRScenes, includes 100k interactive, finely annotated scenes, which can be freely combined into city-scale environments. In contrast to previous works mainly focusing on home, GRScenes covers 89 diverse scene categories, bridging the gap of service-oriented environments where general robots would be initially deployed. (b) GRResidents, a Large Language Model (LLM) driven Non-Player Character (NPC) system that is responsible for social interaction, task generation, and task assignment, thus simulating social scenarios for embodied AI applications. (c) The benchmark, GRBench, supports various robots but focuses on legged robots as primary agents and poses moderately challenging tasks involving Object Loco-Navigation, Social Loco-Navigation, and Loco-Manipulation. We hope that this work can alleviate the scarcity of high-quality data in this field and provide a more comprehensive assessment of Embodied AI research. The project is available at https://github.com/OpenRobotLab/GRUtopia. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_10943 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GRUtopia: Dream General Robots in a City at Scale Wang, Hanqing Chen, Jiahe Huang, Wensi Ben, Qingwei Wang, Tai Mi, Boyu Huang, Tao Zhao, Siheng Chen, Yilun Yang, Sizhe Cao, Peizhou Yu, Wenye Ye, Zichao Li, Jialun Long, Junfeng Wang, Zirui Wang, Huiling Zhao, Ying Tu, Zhongying Qiao, Yu Lin, Dahua Pang, Jiangmiao Robotics Computer Vision and Pattern Recognition Recent works have been exploring the scaling laws in the field of Embodied AI. Given the prohibitive costs of collecting real-world data, we believe the Simulation-to-Real (Sim2Real) paradigm is a crucial step for scaling the learning of embodied models. This paper introduces project GRUtopia, the first simulated interactive 3D society designed for various robots. It features several advancements: (a) The scene dataset, GRScenes, includes 100k interactive, finely annotated scenes, which can be freely combined into city-scale environments. In contrast to previous works mainly focusing on home, GRScenes covers 89 diverse scene categories, bridging the gap of service-oriented environments where general robots would be initially deployed. (b) GRResidents, a Large Language Model (LLM) driven Non-Player Character (NPC) system that is responsible for social interaction, task generation, and task assignment, thus simulating social scenarios for embodied AI applications. (c) The benchmark, GRBench, supports various robots but focuses on legged robots as primary agents and poses moderately challenging tasks involving Object Loco-Navigation, Social Loco-Navigation, and Loco-Manipulation. We hope that this work can alleviate the scarcity of high-quality data in this field and provide a more comprehensive assessment of Embodied AI research. The project is available at https://github.com/OpenRobotLab/GRUtopia. |
| title | GRUtopia: Dream General Robots in a City at Scale |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.10943 |