PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916030968758272 |
|---|---|
| author | Zhang, Yiqi Jiao, Fangzheng Tang, Tian Tian, Boyu Wang, Hangyu Chen, Qiaoling Wang, Guoteng Jiang, Zhen Sun, Peng Zhang, Ping Hu, Xiaohe Liu, Ziming Zhang, Menghao Jia, Yanmin You, Yang Feng, Siyuan |
| author_facet | Zhang, Yiqi Jiao, Fangzheng Tang, Tian Tian, Boyu Wang, Hangyu Chen, Qiaoling Wang, Guoteng Jiang, Zhen Sun, Peng Zhang, Ping Hu, Xiaohe Liu, Ziming Zhang, Menghao Jia, Yanmin You, Yang Feng, Siyuan |
| contents | Reinforcement learning with verifiable rewards (RLVR) has recently unlocked strong reasoning capabilities in large language models (LLMs), triggering rapid exploration of new algorithms and data. However, RLVR training is notoriously inefficient: long-tailed rollouts, tool-induced stalls, and asymmetric resource requirements between rollout and training introduce substantial idle time that cannot be eliminated by job-local optimizations such as synchronous pipelining, asynchronous rollout, or colocated execution.
We argue that this inefficiency is structural. While idle gaps are unavoidable within individual RLVR jobs, they are largely anti-correlated across jobs and therefore exploitable at the cluster level. Leveraging this observation, we present PlexRL, a cluster-level runtime for multiplexing unified LLM services across RLVR jobs. By centrally managing model placement, state transitions, and function-level scheduling under strict affinity constraints, PlexRL time-slices LLM execution across jobs to fill otherwise idle periods without expensive model migration. Our implementation and evaluations demonstrate that PlexRL significantly improves effective cluster capacity and reduces user GPU hour cost by maximum 37.58% while preserving algorithmic flexibility and introducing minimal per-job overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_20863 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR Zhang, Yiqi Jiao, Fangzheng Tang, Tian Tian, Boyu Wang, Hangyu Chen, Qiaoling Wang, Guoteng Jiang, Zhen Sun, Peng Zhang, Ping Hu, Xiaohe Liu, Ziming Zhang, Menghao Jia, Yanmin You, Yang Feng, Siyuan Distributed, Parallel, and Cluster Computing Machine Learning Reinforcement learning with verifiable rewards (RLVR) has recently unlocked strong reasoning capabilities in large language models (LLMs), triggering rapid exploration of new algorithms and data. However, RLVR training is notoriously inefficient: long-tailed rollouts, tool-induced stalls, and asymmetric resource requirements between rollout and training introduce substantial idle time that cannot be eliminated by job-local optimizations such as synchronous pipelining, asynchronous rollout, or colocated execution. We argue that this inefficiency is structural. While idle gaps are unavoidable within individual RLVR jobs, they are largely anti-correlated across jobs and therefore exploitable at the cluster level. Leveraging this observation, we present PlexRL, a cluster-level runtime for multiplexing unified LLM services across RLVR jobs. By centrally managing model placement, state transitions, and function-level scheduling under strict affinity constraints, PlexRL time-slices LLM execution across jobs to fill otherwise idle periods without expensive model migration. Our implementation and evaluations demonstrate that PlexRL significantly improves effective cluster capacity and reduces user GPU hour cost by maximum 37.58% while preserving algorithmic flexibility and introducing minimal per-job overhead. |
| title | PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR |
| topic | Distributed, Parallel, and Cluster Computing Machine Learning |
| url | https://arxiv.org/abs/2605.20863 |