Thousand-GPU Large-Scale Training and Optimization Recipe for AI-Native Cloud Embodied Intelligence Infrastructure
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908899273080832 |
|---|---|
| author | Guo, Yongjian Ma, Yunxuan Sun, Haoran Guan, Zhong Di, Shuai Long, Jing Xu, Wanting Bai, Xiaodong Huang, Wen Guo, Yucheng Zhou, Chen Yang, Qiming Luo, Mingxi Zhao, Tianyun Yang, Hedan Wang, Song Tian, Xiaomeng Xiang, Xiaolong Sun, Zhen Wei, Yu Wang, Luqiao Li, Yuzhen Gu, Chenfeng Xiong, Junwu Gong, Yicheng |
| author_facet | Guo, Yongjian Ma, Yunxuan Sun, Haoran Guan, Zhong Di, Shuai Long, Jing Xu, Wanting Bai, Xiaodong Huang, Wen Guo, Yucheng Zhou, Chen Yang, Qiming Luo, Mingxi Zhao, Tianyun Yang, Hedan Wang, Song Tian, Xiaomeng Xiang, Xiaolong Sun, Zhen Wei, Yu Wang, Luqiao Li, Yuzhen Gu, Chenfeng Xiong, Junwu Gong, Yicheng |
| contents | Embodied intelligence is a key step towards Artificial General Intelligence (AGI), yet its development faces multiple challenges including data, frameworks, infrastructure, and evaluation systems. To address these issues, we have, for the first time in the industry, launched a cloud-based, thousand-GPU distributed training platform for embodied intelligence, built upon the widely adopted LeRobot framework, and have systematically overcome bottlenecks across the entire pipeline. At the data layer, we have restructured the data pipeline to optimize the flow of embodied training data. In terms of training, for the GR00T-N1.5 model, utilizing thousand-GPU clusters and data at the scale of hundreds of millions, the single-round training time has been reduced from 15 hours to just 22 minutes, achieving a 40-fold speedup. At the model layer, by combining variable-length FlashAttention and Data Packing, we have moved from sample redundancy to sequence integration, resulting in a 188% speed increase; π-0.5 attention optimization has accelerated training by 165%; and FP8 quantization has delivered a 140% speedup. On the infrastructure side, relying on high-performance storage, a 3.2T RDMA network, and a Ray-driven elastic AI data lake, we have achieved deep synergy among data, storage, communication, and computation. We have also built an end-to-end evaluation system, creating a closed loop from training to simulation to assessment. This framework has already been fully validated on thousand-GPU clusters, laying a crucial technical foundation for the development and application of next-generation autonomous intelligent robots, and is expected to accelerate the arrival of the era of human-machine integration. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_11101 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Thousand-GPU Large-Scale Training and Optimization Recipe for AI-Native Cloud Embodied Intelligence Infrastructure Guo, Yongjian Ma, Yunxuan Sun, Haoran Guan, Zhong Di, Shuai Long, Jing Xu, Wanting Bai, Xiaodong Huang, Wen Guo, Yucheng Zhou, Chen Yang, Qiming Luo, Mingxi Zhao, Tianyun Yang, Hedan Wang, Song Tian, Xiaomeng Xiang, Xiaolong Sun, Zhen Wei, Yu Wang, Luqiao Li, Yuzhen Gu, Chenfeng Xiong, Junwu Gong, Yicheng Robotics Artificial Intelligence Distributed, Parallel, and Cluster Computing Embodied intelligence is a key step towards Artificial General Intelligence (AGI), yet its development faces multiple challenges including data, frameworks, infrastructure, and evaluation systems. To address these issues, we have, for the first time in the industry, launched a cloud-based, thousand-GPU distributed training platform for embodied intelligence, built upon the widely adopted LeRobot framework, and have systematically overcome bottlenecks across the entire pipeline. At the data layer, we have restructured the data pipeline to optimize the flow of embodied training data. In terms of training, for the GR00T-N1.5 model, utilizing thousand-GPU clusters and data at the scale of hundreds of millions, the single-round training time has been reduced from 15 hours to just 22 minutes, achieving a 40-fold speedup. At the model layer, by combining variable-length FlashAttention and Data Packing, we have moved from sample redundancy to sequence integration, resulting in a 188% speed increase; π-0.5 attention optimization has accelerated training by 165%; and FP8 quantization has delivered a 140% speedup. On the infrastructure side, relying on high-performance storage, a 3.2T RDMA network, and a Ray-driven elastic AI data lake, we have achieved deep synergy among data, storage, communication, and computation. We have also built an end-to-end evaluation system, creating a closed loop from training to simulation to assessment. This framework has already been fully validated on thousand-GPU clusters, laying a crucial technical foundation for the development and application of next-generation autonomous intelligent robots, and is expected to accelerate the arrival of the era of human-machine integration. |
| title | Thousand-GPU Large-Scale Training and Optimization Recipe for AI-Native Cloud Embodied Intelligence Infrastructure |
| topic | Robotics Artificial Intelligence Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2603.11101 |