KARL: Knowledge Agents via Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911489080688640 |
|---|---|
| author | Chang, Jonathan D. Drozdov, Andrew Toshniwal, Shubham Oertell, Owen Trott, Alexander Portes, Jacob Gupta, Abhay Koppol, Pallavi Baheti, Ashutosh Kulinski, Sean Zhou, Ivan Dea, Irene Opsahl-Ong, Krista Favreau-Lessard, Simon Owen, Sean Ortiz, Jose Javier Gonzalez Singhvi, Arnav Andrade, Xabi Wang, Cindy Sreenivasan, Kartik Havens, Sam Liu, Jialu DeNiro, Peyton Sun, Wen Bendersky, Michael Frankle, Jonathan |
| author_facet | Chang, Jonathan D. Drozdov, Andrew Toshniwal, Shubham Oertell, Owen Trott, Alexander Portes, Jacob Gupta, Abhay Koppol, Pallavi Baheti, Ashutosh Kulinski, Sean Zhou, Ivan Dea, Irene Opsahl-Ong, Krista Favreau-Lessard, Simon Owen, Sean Ortiz, Jose Javier Gonzalez Singhvi, Arnav Andrade, Xabi Wang, Cindy Sreenivasan, Kartik Havens, Sam Liu, Jialu DeNiro, Peyton Sun, Wen Bendersky, Michael Frankle, Jonathan |
| contents | We present a system for training enterprise search agents via reinforcement learning that achieves state-of-the-art performance across a diverse suite of hard-to-verify agentic search tasks. Our work makes four core contributions. First, we introduce KARLBench, a multi-capability evaluation suite spanning six distinct search regimes, including constraint-driven entity search, cross-document report synthesis, tabular numerical reasoning, exhaustive entity retrieval, procedural reasoning over technical documentation, and fact aggregation over internal enterprise notes. Second, we show that models trained across heterogeneous search behaviors generalize substantially better than those optimized for any single benchmark. Third, we develop an agentic synthesis pipeline that employs long-horizon reasoning and tool use to generate diverse, grounded, and high-quality training data, with iterative bootstrapping from increasingly capable models. Fourth, we propose a new post-training paradigm based on iterative large-batch off-policy RL that is sample efficient, robust to train-inference engine discrepancies, and naturally extends to multi-task training with out-of-distribution generalization. Compared to Claude 4.6 and GPT 5.2, KARL is Pareto-optimal on KARLBench across cost-quality and latency-quality trade-offs, including tasks that were out-of-distribution during training. With sufficient test-time compute, it surpasses the strongest closed models. These results show that tailored synthetic data in combination with multi-task reinforcement learning enables cost-efficient and high-performing knowledge agents for grounded reasoning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_05218 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | KARL: Knowledge Agents via Reinforcement Learning Chang, Jonathan D. Drozdov, Andrew Toshniwal, Shubham Oertell, Owen Trott, Alexander Portes, Jacob Gupta, Abhay Koppol, Pallavi Baheti, Ashutosh Kulinski, Sean Zhou, Ivan Dea, Irene Opsahl-Ong, Krista Favreau-Lessard, Simon Owen, Sean Ortiz, Jose Javier Gonzalez Singhvi, Arnav Andrade, Xabi Wang, Cindy Sreenivasan, Kartik Havens, Sam Liu, Jialu DeNiro, Peyton Sun, Wen Bendersky, Michael Frankle, Jonathan Artificial Intelligence Machine Learning We present a system for training enterprise search agents via reinforcement learning that achieves state-of-the-art performance across a diverse suite of hard-to-verify agentic search tasks. Our work makes four core contributions. First, we introduce KARLBench, a multi-capability evaluation suite spanning six distinct search regimes, including constraint-driven entity search, cross-document report synthesis, tabular numerical reasoning, exhaustive entity retrieval, procedural reasoning over technical documentation, and fact aggregation over internal enterprise notes. Second, we show that models trained across heterogeneous search behaviors generalize substantially better than those optimized for any single benchmark. Third, we develop an agentic synthesis pipeline that employs long-horizon reasoning and tool use to generate diverse, grounded, and high-quality training data, with iterative bootstrapping from increasingly capable models. Fourth, we propose a new post-training paradigm based on iterative large-batch off-policy RL that is sample efficient, robust to train-inference engine discrepancies, and naturally extends to multi-task training with out-of-distribution generalization. Compared to Claude 4.6 and GPT 5.2, KARL is Pareto-optimal on KARLBench across cost-quality and latency-quality trade-offs, including tasks that were out-of-distribution during training. With sufficient test-time compute, it surpasses the strongest closed models. These results show that tailored synthetic data in combination with multi-task reinforcement learning enables cost-efficient and high-performing knowledge agents for grounded reasoning. |
| title | KARL: Knowledge Agents via Reinforcement Learning |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2603.05218 |