EARL: Efficient Agentic Reinforcement Learning Systems for Large Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915536554688512 |
|---|---|
| author | Tan, Zheyue Abdullahi, Mustapha Shi, Tuo Yuan, Huining Xu, Zelai Yu, Chao Li, Boxun Zhao, Bo |
| author_facet | Tan, Zheyue Abdullahi, Mustapha Shi, Tuo Yuan, Huining Xu, Zelai Yu, Chao Li, Boxun Zhao, Bo |
| contents | Reinforcement learning (RL) has become a pivotal component of large language model (LLM) post-training, and agentic RL extends this paradigm to operate as agents through multi-turn interaction and tool use. Scaling such systems exposes two practical bottlenecks: (1) context length grows rapidly during training, inflating memory usage and latency, and triggering out-of-memory (OOM) failures; and (2) intermediate tensors accumulate with context length, making cross-device data movement a major system bottleneck.
We present EARL, a scalable system for efficient agentic RL. EARL designs a parallelism selector that dynamically adapts model and training parallelism across RL stages based on sequence length and system load, and a data dispatcher that performs layout-aware, decentralized exchange of intermediate data batches. Together, these components increase throughput, reduce long-context failures, and enable stable large-scale training of agentic LLMs without relying on hard limits or penalties of context length. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_05943 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EARL: Efficient Agentic Reinforcement Learning Systems for Large Language Models Tan, Zheyue Abdullahi, Mustapha Shi, Tuo Yuan, Huining Xu, Zelai Yu, Chao Li, Boxun Zhao, Bo Distributed, Parallel, and Cluster Computing Machine Learning Reinforcement learning (RL) has become a pivotal component of large language model (LLM) post-training, and agentic RL extends this paradigm to operate as agents through multi-turn interaction and tool use. Scaling such systems exposes two practical bottlenecks: (1) context length grows rapidly during training, inflating memory usage and latency, and triggering out-of-memory (OOM) failures; and (2) intermediate tensors accumulate with context length, making cross-device data movement a major system bottleneck. We present EARL, a scalable system for efficient agentic RL. EARL designs a parallelism selector that dynamically adapts model and training parallelism across RL stages based on sequence length and system load, and a data dispatcher that performs layout-aware, decentralized exchange of intermediate data batches. Together, these components increase throughput, reduce long-context failures, and enable stable large-scale training of agentic LLMs without relying on hard limits or penalties of context length. |
| title | EARL: Efficient Agentic Reinforcement Learning Systems for Large Language Models |
| topic | Distributed, Parallel, and Cluster Computing Machine Learning |
| url | https://arxiv.org/abs/2510.05943 |