Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.05789 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909419367825408 |
|---|---|
| author | Ren, Pengzhen Li, Min Luo, Zhen Song, Xinshuai Chen, Ziwei Liufu, Weijia Yang, Yixuan Zheng, Hao Xu, Rongtao Huang, Zitong Ding, Tongsheng Xie, Luyang Zhang, Kaidong Fu, Changfei Liu, Yang Lin, Liang Zheng, Feng Liang, Xiaodan |
| author_facet | Ren, Pengzhen Li, Min Luo, Zhen Song, Xinshuai Chen, Ziwei Liufu, Weijia Yang, Yixuan Zheng, Hao Xu, Rongtao Huang, Zitong Ding, Tongsheng Xie, Luyang Zhang, Kaidong Fu, Changfei Liu, Yang Lin, Liang Zheng, Feng Liang, Xiaodan |
| contents | Realizing scaling laws in embodied AI has become a focus. However, previous work has been scattered across diverse simulation platforms, with assets and models lacking unified interfaces, which has led to inefficiencies in research. To address this, we introduce InfiniteWorld, a unified and scalable simulator for general vision-language robot interaction built on Nvidia Isaac Sim. InfiniteWorld encompasses a comprehensive set of physics asset construction methods and generalized free robot interaction benchmarks. Specifically, we first built a unified and scalable simulation framework for embodied learning that integrates a series of improvements in generation-driven 3D asset construction, Real2Sim, automated annotation framework, and unified 3D asset processing. This framework provides a unified and scalable platform for robot interaction and learning. In addition, to simulate realistic robot interaction, we build four new general benchmarks, including scene graph collaborative exploration and open-world social mobile manipulation. The former is often overlooked as an important task for robots to explore the environment and build scene knowledge, while the latter simulates robot interaction tasks with different levels of knowledge agents based on the former. They can more comprehensively evaluate the embodied agent's capabilities in environmental understanding, task planning and execution, and intelligent interaction. We hope that this work can provide the community with a systematic asset interface, alleviate the dilemma of the lack of high-quality assets, and provide a more comprehensive evaluation of robot interactions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_05789 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | InfiniteWorld: A Unified Scalable Simulation Framework for General Visual-Language Robot Interaction Ren, Pengzhen Li, Min Luo, Zhen Song, Xinshuai Chen, Ziwei Liufu, Weijia Yang, Yixuan Zheng, Hao Xu, Rongtao Huang, Zitong Ding, Tongsheng Xie, Luyang Zhang, Kaidong Fu, Changfei Liu, Yang Lin, Liang Zheng, Feng Liang, Xiaodan Robotics Realizing scaling laws in embodied AI has become a focus. However, previous work has been scattered across diverse simulation platforms, with assets and models lacking unified interfaces, which has led to inefficiencies in research. To address this, we introduce InfiniteWorld, a unified and scalable simulator for general vision-language robot interaction built on Nvidia Isaac Sim. InfiniteWorld encompasses a comprehensive set of physics asset construction methods and generalized free robot interaction benchmarks. Specifically, we first built a unified and scalable simulation framework for embodied learning that integrates a series of improvements in generation-driven 3D asset construction, Real2Sim, automated annotation framework, and unified 3D asset processing. This framework provides a unified and scalable platform for robot interaction and learning. In addition, to simulate realistic robot interaction, we build four new general benchmarks, including scene graph collaborative exploration and open-world social mobile manipulation. The former is often overlooked as an important task for robots to explore the environment and build scene knowledge, while the latter simulates robot interaction tasks with different levels of knowledge agents based on the former. They can more comprehensively evaluate the embodied agent's capabilities in environmental understanding, task planning and execution, and intelligent interaction. We hope that this work can provide the community with a systematic asset interface, alleviate the dilemma of the lack of high-quality assets, and provide a more comprehensive evaluation of robot interactions. |
| title | InfiniteWorld: A Unified Scalable Simulation Framework for General Visual-Language Robot Interaction |
| topic | Robotics |
| url | https://arxiv.org/abs/2412.05789 |