AIRA_2: Overcoming Bottlenecks in AI Research Agents
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908959323979776 |
|---|---|
| author | Hambardzumyan, Karen Baldwin, Nicolas Toledo, Edan Hazra, Rishi Kuchnik, Michael Omari, Bassel Al Foster, Thomas Simon Protopopov, Anton Gagnon-Audet, Jean-Christophe Mediratta, Ishita Niu, Kelvin Shvartsman, Michael Lupidi, Alisia Audran-Reiss, Alexis Pathak, Parth Shavrina, Tatiana Magka, Despoina Momand, Hela Dunfield, Derek Cancedda, Nicola Stenetorp, Pontus Wu, Carole-Jean Foerster, Jakob Nicolaus Bachrach, Yoram Josifoski, Martin |
| author_facet | Hambardzumyan, Karen Baldwin, Nicolas Toledo, Edan Hazra, Rishi Kuchnik, Michael Omari, Bassel Al Foster, Thomas Simon Protopopov, Anton Gagnon-Audet, Jean-Christophe Mediratta, Ishita Niu, Kelvin Shvartsman, Michael Lupidi, Alisia Audran-Reiss, Alexis Pathak, Parth Shavrina, Tatiana Magka, Despoina Momand, Hela Dunfield, Derek Cancedda, Nicola Stenetorp, Pontus Wu, Carole-Jean Foerster, Jakob Nicolaus Bachrach, Yoram Josifoski, Martin |
| contents | Existing research has identified three structural performance bottlenecks in AI research agents: (1) synchronous single-GPU execution constrains sample throughput, limiting the benefit of search; (2) a generalization gap where validation-based selection causes overfitting and performance to degrade over extended search horizons; and (3) the limited capability of fixed, single-turn LLM operators imposes a ceiling on search performance. We introduce AIRA$_2$, which addresses these bottlenecks through three architectural choices: an asynchronous multi-GPU worker pool that increases experiment throughput linearly; a Hidden Consistent Evaluation protocol that delivers a reliable evaluation signal; and ReAct agents that dynamically scope their actions and debug interactively. On MLE-bench-30, AIRA$^{\dagger}_{2}$ achieves a mean Percentile Rank of 81.5% at 24 hours and 83.1% at 72 hours, outperforming the strongest baseline, which achieves 72.7%. On AIRS-Bench, AIRA$_2$ exceeds human state-of-the-art on 6 out of 20 diverse research tasks. Ablations confirm that each architectural component is necessary, that performance follows a predictable scaling law that transfers across LLM backbones, and that the "overfitting" reported in prior work was driven by evaluation noise rather than true data memorization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_26499 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AIRA_2: Overcoming Bottlenecks in AI Research Agents Hambardzumyan, Karen Baldwin, Nicolas Toledo, Edan Hazra, Rishi Kuchnik, Michael Omari, Bassel Al Foster, Thomas Simon Protopopov, Anton Gagnon-Audet, Jean-Christophe Mediratta, Ishita Niu, Kelvin Shvartsman, Michael Lupidi, Alisia Audran-Reiss, Alexis Pathak, Parth Shavrina, Tatiana Magka, Despoina Momand, Hela Dunfield, Derek Cancedda, Nicola Stenetorp, Pontus Wu, Carole-Jean Foerster, Jakob Nicolaus Bachrach, Yoram Josifoski, Martin Artificial Intelligence Existing research has identified three structural performance bottlenecks in AI research agents: (1) synchronous single-GPU execution constrains sample throughput, limiting the benefit of search; (2) a generalization gap where validation-based selection causes overfitting and performance to degrade over extended search horizons; and (3) the limited capability of fixed, single-turn LLM operators imposes a ceiling on search performance. We introduce AIRA$_2$, which addresses these bottlenecks through three architectural choices: an asynchronous multi-GPU worker pool that increases experiment throughput linearly; a Hidden Consistent Evaluation protocol that delivers a reliable evaluation signal; and ReAct agents that dynamically scope their actions and debug interactively. On MLE-bench-30, AIRA$^{\dagger}_{2}$ achieves a mean Percentile Rank of 81.5% at 24 hours and 83.1% at 72 hours, outperforming the strongest baseline, which achieves 72.7%. On AIRS-Bench, AIRA$_2$ exceeds human state-of-the-art on 6 out of 20 diverse research tasks. Ablations confirm that each architectural component is necessary, that performance follows a predictable scaling law that transfers across LLM backbones, and that the "overfitting" reported in prior work was driven by evaluation noise rather than true data memorization. |
| title | AIRA_2: Overcoming Bottlenecks in AI Research Agents |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2603.26499 |