RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917256231911424 |
|---|---|
| author | Zang, Hongzhi Wei, Mingjie Xu, Si Wu, Yongji Guo, Zhen Wang, Yuanqing Lin, Hao Wang, Peihong Shi, Liangzhi Xie, Yuqing Xu, Zhexuan Liu, Zhihao Chen, Kang Tang, Wenhao Zhang, Quanlu Zhang, Weinan Yu, Chao Wang, Yu |
| author_facet | Zang, Hongzhi Wei, Mingjie Xu, Si Wu, Yongji Guo, Zhen Wang, Yuanqing Lin, Hao Wang, Peihong Shi, Liangzhi Xie, Yuqing Xu, Zhexuan Liu, Zhihao Chen, Kang Tang, Wenhao Zhang, Quanlu Zhang, Weinan Yu, Chao Wang, Yu |
| contents | Recent advances in vision-language-action (VLA) models have motivated the extension of their capabilities to embodied settings, where reinforcement learning (RL) offers a principled way to optimize task success through interaction. However, existing methods remain fragmented, lacking both a unified platform for fair comparison across architectures and algorithms and an efficient system design for scalable training. To address these challenges, we introduce RLinf-VLA, a unified and efficient framework for scalable RL training of VLA models. RLinf-VLA achieves unification by providing a unified interface that standardizes the integration of diverse VLA architectures, multiple RL algorithms, and heterogeneous simulators, enabling extensibility. To ensure efficiency, the system adopts a flexible resource allocation architecture for rendering, inference, and training workloads in RL pipelines. In particular, for GPU-parallelized simulators, RLinf-VLA introduces a hybrid fine-grained pipeline allocation strategy, yielding a 1.61x-1.88x training speedup. Using this unified system, models trained with RLinf-VLA demonstrate consistent performance improvements of approximately 20-85% across multiple simulation benchmarks, including LIBERO, ManiSkill, and RoboTwin. Furthermore, we distill a set of training practices for effective RL-based VLA training. We position RLinf-VLA as a foundational system to enable efficient, unified, and reproducible research in embodied intelligence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_06710 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Models Zang, Hongzhi Wei, Mingjie Xu, Si Wu, Yongji Guo, Zhen Wang, Yuanqing Lin, Hao Wang, Peihong Shi, Liangzhi Xie, Yuqing Xu, Zhexuan Liu, Zhihao Chen, Kang Tang, Wenhao Zhang, Quanlu Zhang, Weinan Yu, Chao Wang, Yu Robotics Recent advances in vision-language-action (VLA) models have motivated the extension of their capabilities to embodied settings, where reinforcement learning (RL) offers a principled way to optimize task success through interaction. However, existing methods remain fragmented, lacking both a unified platform for fair comparison across architectures and algorithms and an efficient system design for scalable training. To address these challenges, we introduce RLinf-VLA, a unified and efficient framework for scalable RL training of VLA models. RLinf-VLA achieves unification by providing a unified interface that standardizes the integration of diverse VLA architectures, multiple RL algorithms, and heterogeneous simulators, enabling extensibility. To ensure efficiency, the system adopts a flexible resource allocation architecture for rendering, inference, and training workloads in RL pipelines. In particular, for GPU-parallelized simulators, RLinf-VLA introduces a hybrid fine-grained pipeline allocation strategy, yielding a 1.61x-1.88x training speedup. Using this unified system, models trained with RLinf-VLA demonstrate consistent performance improvements of approximately 20-85% across multiple simulation benchmarks, including LIBERO, ManiSkill, and RoboTwin. Furthermore, we distill a set of training practices for effective RL-based VLA training. We position RLinf-VLA as a foundational system to enable efficient, unified, and reproducible research in embodied intelligence. |
| title | RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Models |
| topic | Robotics |
| url | https://arxiv.org/abs/2510.06710 |