Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.07837 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912898489516032 |
|---|---|
| author | Zang, Hongzhi Yu, Shu'ang Lin, Hao Zhou, Tianxing Huang, Zefang Guo, Zhen Xu, Xin Zhou, Jiakai Sheng, Yuze Zhang, Shizhe Gao, Feng Tang, Wenhao Yue, Yufeng Zhang, Quanlu Chen, Xinlei Yu, Chao Wang, Yu |
| author_facet | Zang, Hongzhi Yu, Shu'ang Lin, Hao Zhou, Tianxing Huang, Zefang Guo, Zhen Xu, Xin Zhou, Jiakai Sheng, Yuze Zhang, Shizhe Gao, Feng Tang, Wenhao Yue, Yufeng Zhang, Quanlu Chen, Xinlei Yu, Chao Wang, Yu |
| contents | Online policy learning directly in the physical world is a promising yet challenging direction for embodied intelligence. Unlike simulation, real-world systems cannot be arbitrarily accelerated, cheaply reset, or massively replicated, which makes scalable data collection, heterogeneous deployment, and long-horizon effective training difficult. These challenges suggest that real-world policy learning is not only an algorithmic issue but fundamentally a systems problem. We present USER, a Unified and extensible SystEm for Real-world online policy learning. USER treats physical robots as first-class hardware resources alongside GPUs through a unified hardware abstraction layer, enabling automatic discovery, management, and scheduling of heterogeneous robots. To address cloud-edge communication, USER introduces an adaptive communication plane with tunneling-based networking, distributed data channels for traffic localization, and streaming-multiprocessor-aware weight synchronization to regulate GPU-side overhead. On top of this infrastructure, USER organizes learning as a fully asynchronous framework with a persistent, cache-aware buffer, enabling efficient long-horizon experiments with robust crash recovery and reuse of historical data. In addition, USER provides extensible abstractions for rewards, algorithms, and policies, supporting online imitation or reinforcement learning of CNN/MLP, generative policies, and large vision-language-action (VLA) models within a unified pipeline. Results in both simulation and the real world show that USER enables multi-robot coordination, heterogeneous manipulators, edge-cloud collaboration with large models, and long-running asynchronous training, offering a unified and extensible systems foundation for real-world online policy learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_07837 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | RLinf-USER: A Unified and Extensible System for Real-World Online Policy Learning in Embodied AI Zang, Hongzhi Yu, Shu'ang Lin, Hao Zhou, Tianxing Huang, Zefang Guo, Zhen Xu, Xin Zhou, Jiakai Sheng, Yuze Zhang, Shizhe Gao, Feng Tang, Wenhao Yue, Yufeng Zhang, Quanlu Chen, Xinlei Yu, Chao Wang, Yu Robotics Online policy learning directly in the physical world is a promising yet challenging direction for embodied intelligence. Unlike simulation, real-world systems cannot be arbitrarily accelerated, cheaply reset, or massively replicated, which makes scalable data collection, heterogeneous deployment, and long-horizon effective training difficult. These challenges suggest that real-world policy learning is not only an algorithmic issue but fundamentally a systems problem. We present USER, a Unified and extensible SystEm for Real-world online policy learning. USER treats physical robots as first-class hardware resources alongside GPUs through a unified hardware abstraction layer, enabling automatic discovery, management, and scheduling of heterogeneous robots. To address cloud-edge communication, USER introduces an adaptive communication plane with tunneling-based networking, distributed data channels for traffic localization, and streaming-multiprocessor-aware weight synchronization to regulate GPU-side overhead. On top of this infrastructure, USER organizes learning as a fully asynchronous framework with a persistent, cache-aware buffer, enabling efficient long-horizon experiments with robust crash recovery and reuse of historical data. In addition, USER provides extensible abstractions for rewards, algorithms, and policies, supporting online imitation or reinforcement learning of CNN/MLP, generative policies, and large vision-language-action (VLA) models within a unified pipeline. Results in both simulation and the real world show that USER enables multi-robot coordination, heterogeneous manipulators, edge-cloud collaboration with large models, and long-running asynchronous training, offering a unified and extensible systems foundation for real-world online policy learning. |
| title | RLinf-USER: A Unified and Extensible System for Real-World Online Policy Learning in Embodied AI |
| topic | Robotics |
| url | https://arxiv.org/abs/2602.07837 |