AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917058858450944 |
|---|---|
| author | Toledo, Edan Hambardzumyan, Karen Josifoski, Martin Hazra, Rishi Baldwin, Nicolas Audran-Reiss, Alexis Kuchnik, Michael Magka, Despoina Jiang, Minqi Lupidi, Alisia Maria Lupu, Andrei Raileanu, Roberta Niu, Kelvin Shavrina, Tatiana Gagnon-Audet, Jean-Christophe Shvartsman, Michael Sodhani, Shagun Miller, Alexander H. Charnalia, Abhishek Dunfield, Derek Wu, Carole-Jean Stenetorp, Pontus Cancedda, Nicola Foerster, Jakob Nicolaus Bachrach, Yoram |
| author_facet | Toledo, Edan Hambardzumyan, Karen Josifoski, Martin Hazra, Rishi Baldwin, Nicolas Audran-Reiss, Alexis Kuchnik, Michael Magka, Despoina Jiang, Minqi Lupidi, Alisia Maria Lupu, Andrei Raileanu, Roberta Niu, Kelvin Shavrina, Tatiana Gagnon-Audet, Jean-Christophe Shvartsman, Michael Sodhani, Shagun Miller, Alexander H. Charnalia, Abhishek Dunfield, Derek Wu, Carole-Jean Stenetorp, Pontus Cancedda, Nicola Foerster, Jakob Nicolaus Bachrach, Yoram |
| contents | AI research agents are demonstrating great potential to accelerate scientific progress by automating the design, implementation, and training of machine learning models. We focus on methods for improving agents' performance on MLE-bench, a challenging benchmark where agents compete in Kaggle competitions to solve real-world machine learning problems. We formalize AI research agents as search policies that navigate a space of candidate solutions, iteratively modifying them using operators. By designing and systematically varying different operator sets and search policies (Greedy, MCTS, Evolutionary), we show that their interplay is critical for achieving high performance. Our best pairing of search strategy and operator set achieves a state-of-the-art result on MLE-bench lite, increasing the success rate of achieving a Kaggle medal from 39.6% to 47.7%. Our investigation underscores the importance of jointly considering the search strategy, operator design, and evaluation methodology in advancing automated machine learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_02554 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench Toledo, Edan Hambardzumyan, Karen Josifoski, Martin Hazra, Rishi Baldwin, Nicolas Audran-Reiss, Alexis Kuchnik, Michael Magka, Despoina Jiang, Minqi Lupidi, Alisia Maria Lupu, Andrei Raileanu, Roberta Niu, Kelvin Shavrina, Tatiana Gagnon-Audet, Jean-Christophe Shvartsman, Michael Sodhani, Shagun Miller, Alexander H. Charnalia, Abhishek Dunfield, Derek Wu, Carole-Jean Stenetorp, Pontus Cancedda, Nicola Foerster, Jakob Nicolaus Bachrach, Yoram Artificial Intelligence Machine Learning AI research agents are demonstrating great potential to accelerate scientific progress by automating the design, implementation, and training of machine learning models. We focus on methods for improving agents' performance on MLE-bench, a challenging benchmark where agents compete in Kaggle competitions to solve real-world machine learning problems. We formalize AI research agents as search policies that navigate a space of candidate solutions, iteratively modifying them using operators. By designing and systematically varying different operator sets and search policies (Greedy, MCTS, Evolutionary), we show that their interplay is critical for achieving high performance. Our best pairing of search strategy and operator set achieves a state-of-the-art result on MLE-bench lite, increasing the success rate of achieving a Kaggle medal from 39.6% to 47.7%. Our investigation underscores the importance of jointly considering the search strategy, operator design, and evaluation methodology in advancing automated machine learning. |
| title | AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2507.02554 |