Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.00425 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913866286366720 |
|---|---|
| author | Tao, Stone Xiang, Fanbo Shukla, Arth Qin, Yuzhe Hinrichsen, Xander Yuan, Xiaodi Bao, Chen Lin, Xinsong Liu, Yulin Chan, Tse-kai Gao, Yuan Li, Xuanlin Mu, Tongzhou Xiao, Nan Gurha, Arnav Rajesh, Viswesh Nagaswamy Choi, Yong Woo Chen, Yen-Ru Huang, Zhiao Calandra, Roberto Chen, Rui Luo, Shan Su, Hao |
| author_facet | Tao, Stone Xiang, Fanbo Shukla, Arth Qin, Yuzhe Hinrichsen, Xander Yuan, Xiaodi Bao, Chen Lin, Xinsong Liu, Yulin Chan, Tse-kai Gao, Yuan Li, Xuanlin Mu, Tongzhou Xiao, Nan Gurha, Arnav Rajesh, Viswesh Nagaswamy Choi, Yong Woo Chen, Yen-Ru Huang, Zhiao Calandra, Roberto Chen, Rui Luo, Shan Su, Hao |
| contents | Simulation has enabled unprecedented compute-scalable approaches to robot learning. However, many existing simulation frameworks typically support a narrow range of scenes/tasks and lack features critical for scaling generalizable robotics and sim2real. We introduce and open source ManiSkill3, the fastest state-visual GPU parallelized robotics simulator with contact-rich physics targeting generalizable manipulation. ManiSkill3 supports GPU parallelization of many aspects including simulation+rendering, heterogeneous simulation, pointclouds/voxels visual input, and more. Simulation with rendering on ManiSkill3 can run 10-1000x faster with 2-3x less GPU memory usage than other platforms, achieving up to 30,000+ FPS in benchmarked environments due to minimal python/pytorch overhead in the system, simulation on the GPU, and the use of the SAPIEN parallel rendering system. Tasks that used to take hours to train can now take minutes. We further provide the most comprehensive range of GPU parallelized environments/tasks spanning 12 distinct domains including but not limited to mobile manipulation for tasks such as drawing, humanoids, and dextrous manipulation in realistic scenes designed by artists or real-world digital twins. In addition, millions of demonstration frames are provided from motion planning, RL, and teleoperation. ManiSkill3 also provides a comprehensive set of baselines that span popular RL and learning-from-demonstrations algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_00425 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI Tao, Stone Xiang, Fanbo Shukla, Arth Qin, Yuzhe Hinrichsen, Xander Yuan, Xiaodi Bao, Chen Lin, Xinsong Liu, Yulin Chan, Tse-kai Gao, Yuan Li, Xuanlin Mu, Tongzhou Xiao, Nan Gurha, Arnav Rajesh, Viswesh Nagaswamy Choi, Yong Woo Chen, Yen-Ru Huang, Zhiao Calandra, Roberto Chen, Rui Luo, Shan Su, Hao Robotics Artificial Intelligence Simulation has enabled unprecedented compute-scalable approaches to robot learning. However, many existing simulation frameworks typically support a narrow range of scenes/tasks and lack features critical for scaling generalizable robotics and sim2real. We introduce and open source ManiSkill3, the fastest state-visual GPU parallelized robotics simulator with contact-rich physics targeting generalizable manipulation. ManiSkill3 supports GPU parallelization of many aspects including simulation+rendering, heterogeneous simulation, pointclouds/voxels visual input, and more. Simulation with rendering on ManiSkill3 can run 10-1000x faster with 2-3x less GPU memory usage than other platforms, achieving up to 30,000+ FPS in benchmarked environments due to minimal python/pytorch overhead in the system, simulation on the GPU, and the use of the SAPIEN parallel rendering system. Tasks that used to take hours to train can now take minutes. We further provide the most comprehensive range of GPU parallelized environments/tasks spanning 12 distinct domains including but not limited to mobile manipulation for tasks such as drawing, humanoids, and dextrous manipulation in realistic scenes designed by artists or real-world digital twins. In addition, millions of demonstration frames are provided from motion planning, RL, and teleoperation. ManiSkill3 also provides a comprehensive set of baselines that span popular RL and learning-from-demonstrations algorithms. |
| title | ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2410.00425 |