Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.24220 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914594334703616 |
|---|---|
| author | Xu, Binfeng Zhang, Hao Zhang, Shaokun Han, Songyang Liu, Mingjie Hu, Jian Diao, Shizhe Jin, Zhenghui Zou, Yunheng Demoret, Michael Kautz, Jan Dong, Yi |
| author_facet | Xu, Binfeng Zhang, Hao Zhang, Shaokun Han, Songyang Liu, Mingjie Hu, Jian Diao, Shizhe Jin, Zhenghui Zou, Yunheng Demoret, Michael Kautz, Jan Dong, Yi |
| contents | Reinforcement learning for language agents increasingly depends on custom harnesses that manage long-running context, multi-turn tool use and multi-agent orchestration. However, porting these harnesses into RL environment interfaces remains difficult and often loses important training signals. We bridge this gap with polar, a rollout framework for scalable asynchronous RL over arbitrary agent harnesses. Polar treats the agent harness as a black box: it proxies LLM API calls, records token-level model interactions, and reconstructs token-faithful trajectories for training. Each rollout node efficiently manages runtime prewarming, agent execution, trajectory reconstruction, and evaluation in parallel, exposing asynchronous service endpoints that can be consumed by independent trainers at scale. This decoupled design makes Polar agnostic to agent harnesses, training infrastructure, and RL algorithms while improving compute utilization for long-running agent workloads. We validate polar by training agents on software-engineering tasks with popular coding harnesses. Using simple GRPO, polar improves Qwen3.5-4B by 22.6, 4.8, 0.6 and 6.2 points on SWE-Bench Verified with the Codex, Claude Code, Qwen Code and Pi harnesses, respectively. We further demonstrate Polar for offline data generation over custom harnesses and ablate trajectory reconstruction strategies. Polar rewrites its preceding work, Prorl Agent, and has been registered as one of NeMo Gym environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_24220 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Polar: Agentic RL on Any Harness at Scale Xu, Binfeng Zhang, Hao Zhang, Shaokun Han, Songyang Liu, Mingjie Hu, Jian Diao, Shizhe Jin, Zhenghui Zou, Yunheng Demoret, Michael Kautz, Jan Dong, Yi Distributed, Parallel, and Cluster Computing Reinforcement learning for language agents increasingly depends on custom harnesses that manage long-running context, multi-turn tool use and multi-agent orchestration. However, porting these harnesses into RL environment interfaces remains difficult and often loses important training signals. We bridge this gap with polar, a rollout framework for scalable asynchronous RL over arbitrary agent harnesses. Polar treats the agent harness as a black box: it proxies LLM API calls, records token-level model interactions, and reconstructs token-faithful trajectories for training. Each rollout node efficiently manages runtime prewarming, agent execution, trajectory reconstruction, and evaluation in parallel, exposing asynchronous service endpoints that can be consumed by independent trainers at scale. This decoupled design makes Polar agnostic to agent harnesses, training infrastructure, and RL algorithms while improving compute utilization for long-running agent workloads. We validate polar by training agents on software-engineering tasks with popular coding harnesses. Using simple GRPO, polar improves Qwen3.5-4B by 22.6, 4.8, 0.6 and 6.2 points on SWE-Bench Verified with the Codex, Claude Code, Qwen Code and Pi harnesses, respectively. We further demonstrate Polar for offline data generation over custom harnesses and ablate trajectory reconstruction strategies. Polar rewrites its preceding work, Prorl Agent, and has been registered as one of NeMo Gym environments. |
| title | Polar: Agentic RL on Any Harness at Scale |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2605.24220 |