Breaking the Performance Ceiling in Reinforcement Learning requires Inference Strategies
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908719236775936 |
|---|---|
| author | Chalumeau, Felix Rajaonarivonivelomanantsoa, Daniel de Kock, Ruan Formanek, Claude Abramowitz, Sasha Mahjoub, Oumayma Khlifi, Wiem Toit, Simon Du Nessir, Louay Ben Shabe, Refiloe De Nicola, Noah Fokam, Arnol Singh, Siddarth Sob, Ulrich Mbou Pretorius, Arnu |
| author_facet | Chalumeau, Felix Rajaonarivonivelomanantsoa, Daniel de Kock, Ruan Formanek, Claude Abramowitz, Sasha Mahjoub, Oumayma Khlifi, Wiem Toit, Simon Du Nessir, Louay Ben Shabe, Refiloe De Nicola, Noah Fokam, Arnol Singh, Siddarth Sob, Ulrich Mbou Pretorius, Arnu |
| contents | Reinforcement learning (RL) systems have countless applications, from energy-grid management to protein design. However, such real-world scenarios are often extremely difficult, combinatorial in nature, and require complex coordination between multiple agents. This level of complexity can cause even state-of-the-art RL systems, trained until convergence, to hit a performance ceiling which they are unable to break out of with zero-shot inference. Meanwhile, many digital or simulation-based applications allow for an inference phase that utilises a specific time and compute budget to explore multiple attempts before outputting a final solution. In this work, we show that such an inference phase employed at execution time, and the choice of a corresponding inference strategy, are key to breaking the performance ceiling observed in complex multi-agent RL problems. Our main result is striking: we can obtain up to a 126% and, on average, a 45% improvement over the previous state-of-the-art across 17 tasks, using only a couple seconds of extra wall-clock time during execution. We also demonstrate promising compute scaling properties, supported by over 60k experiments, making it the largest study on inference strategies for complex RL to date. Our experimental data and code are available at https://sites.google.com/view/inference-strategies-rl. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_21236 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Breaking the Performance Ceiling in Reinforcement Learning requires Inference Strategies Chalumeau, Felix Rajaonarivonivelomanantsoa, Daniel de Kock, Ruan Formanek, Claude Abramowitz, Sasha Mahjoub, Oumayma Khlifi, Wiem Toit, Simon Du Nessir, Louay Ben Shabe, Refiloe De Nicola, Noah Fokam, Arnol Singh, Siddarth Sob, Ulrich Mbou Pretorius, Arnu Machine Learning Artificial Intelligence Multiagent Systems Reinforcement learning (RL) systems have countless applications, from energy-grid management to protein design. However, such real-world scenarios are often extremely difficult, combinatorial in nature, and require complex coordination between multiple agents. This level of complexity can cause even state-of-the-art RL systems, trained until convergence, to hit a performance ceiling which they are unable to break out of with zero-shot inference. Meanwhile, many digital or simulation-based applications allow for an inference phase that utilises a specific time and compute budget to explore multiple attempts before outputting a final solution. In this work, we show that such an inference phase employed at execution time, and the choice of a corresponding inference strategy, are key to breaking the performance ceiling observed in complex multi-agent RL problems. Our main result is striking: we can obtain up to a 126% and, on average, a 45% improvement over the previous state-of-the-art across 17 tasks, using only a couple seconds of extra wall-clock time during execution. We also demonstrate promising compute scaling properties, supported by over 60k experiments, making it the largest study on inference strategies for complex RL to date. Our experimental data and code are available at https://sites.google.com/view/inference-strategies-rl. |
| title | Breaking the Performance Ceiling in Reinforcement Learning requires Inference Strategies |
| topic | Machine Learning Artificial Intelligence Multiagent Systems |
| url | https://arxiv.org/abs/2505.21236 |