Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2508.11553 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909738046849024 |
|---|---|
| author | Wang, Jinghui Wang, Shaojie Cui, Yinghan Chen, Xuxing Wang, Chao Zhang, Xiaojiang Zhang, Minglei Zhang, Jiarong Zhuang, Wenhao Cao, Yuchen Bao, Wankang Li, Haimo Lin, Zheng Wang, Huiming Huang, Haoyang Feng, Zongxian Zhan, Zizheng Deng, Ken Xiang, Wen Tang, Huaixi Wu, Kun Li, Mengtong Xie, Mengfei Peng, Junyi Zhang, Haotian Chen, Bin Yu, Bing |
| author_facet | Wang, Jinghui Wang, Shaojie Cui, Yinghan Chen, Xuxing Wang, Chao Zhang, Xiaojiang Zhang, Minglei Zhang, Jiarong Zhuang, Wenhao Cao, Yuchen Bao, Wankang Li, Haimo Lin, Zheng Wang, Huiming Huang, Haoyang Feng, Zongxian Zhan, Zizheng Deng, Ken Xiang, Wen Tang, Huaixi Wu, Kun Li, Mengtong Xie, Mengfei Peng, Junyi Zhang, Haotian Chen, Bin Yu, Bing |
| contents | We introduce SeamlessFlow, a server based reinforcement learning (RL) framework that addresses two core challenges in industrial scale RL: (1) decoupling RL training from the complex execution flow of agents; (2) maximizing GPU utilization with minimal idle time while preserving the stability and scalability required for large-scale deployments. First, SeamlessFlow introduces a data plane that decouples the RL trainer from diverse, complex agent implementations while sustaining high throughput. A central trajectory manager maintains complete interaction histories and supports partial rollout, allowing rollout to pause for weight updates and resume seamlessly, keeping agents unaware of service interruptions. Second, we propose a tag driven scheduling paradigm that abstracts hardware into capability tagged resources, unifying colocated and disaggregated architectures. Based on this, SeamlessFlow introduces a spatiotemporal multiplexing pipeline that dynamically reassigns idle training nodes to rollout in a train rollout separated setup, eliminating pipeline bubbles and fully exploiting heterogeneous cluster resources. By combining these innovations, SeamlessFlow delivers both stability and high performance, making it well suited for multi agent, long horizon, and other complex RL tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_11553 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SeamlessFlow: A Trainer Agent Isolation RL Framework Achieving Bubble-Free Pipelines via Tag Scheduling Wang, Jinghui Wang, Shaojie Cui, Yinghan Chen, Xuxing Wang, Chao Zhang, Xiaojiang Zhang, Minglei Zhang, Jiarong Zhuang, Wenhao Cao, Yuchen Bao, Wankang Li, Haimo Lin, Zheng Wang, Huiming Huang, Haoyang Feng, Zongxian Zhan, Zizheng Deng, Ken Xiang, Wen Tang, Huaixi Wu, Kun Li, Mengtong Xie, Mengfei Peng, Junyi Zhang, Haotian Chen, Bin Yu, Bing Machine Learning We introduce SeamlessFlow, a server based reinforcement learning (RL) framework that addresses two core challenges in industrial scale RL: (1) decoupling RL training from the complex execution flow of agents; (2) maximizing GPU utilization with minimal idle time while preserving the stability and scalability required for large-scale deployments. First, SeamlessFlow introduces a data plane that decouples the RL trainer from diverse, complex agent implementations while sustaining high throughput. A central trajectory manager maintains complete interaction histories and supports partial rollout, allowing rollout to pause for weight updates and resume seamlessly, keeping agents unaware of service interruptions. Second, we propose a tag driven scheduling paradigm that abstracts hardware into capability tagged resources, unifying colocated and disaggregated architectures. Based on this, SeamlessFlow introduces a spatiotemporal multiplexing pipeline that dynamically reassigns idle training nodes to rollout in a train rollout separated setup, eliminating pipeline bubbles and fully exploiting heterogeneous cluster resources. By combining these innovations, SeamlessFlow delivers both stability and high performance, making it well suited for multi agent, long horizon, and other complex RL tasks. |
| title | SeamlessFlow: A Trainer Agent Isolation RL Framework Achieving Bubble-Free Pipelines via Tag Scheduling |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2508.11553 |