MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909756599304192 |
|---|---|
| author | Nam, Jaehyun Yoon, Jinsung Chen, Jiefeng Shin, Jinwoo Arık, Sercan Ö. Pfister, Tomas |
| author_facet | Nam, Jaehyun Yoon, Jinsung Chen, Jiefeng Shin, Jinwoo Arık, Sercan Ö. Pfister, Tomas |
| contents | Agents based on large language models (LLMs) for machine learning engineering (MLE) can automatically implement ML models via code generation. However, existing approaches to build such agents often rely heavily on inherent LLM knowledge and employ coarse exploration strategies that modify the entire code structure at once. This limits their ability to select effective task-specific models and perform deep exploration within specific components, such as experimenting extensively with feature engineering options. To overcome these, we propose MLE-STAR, a novel approach to build MLE agents. MLE-STAR first leverages external knowledge by using a search engine to retrieve effective models from the web, forming an initial solution, then iteratively refines it by exploring various strategies targeting specific ML components. This exploration is guided by ablation studies analyzing the impact of individual code blocks. Furthermore, we introduce a novel ensembling method using an effective strategy suggested by MLE-STAR. Our experimental results show that MLE-STAR achieves medals in 64% of the Kaggle competitions on the MLE-bench Lite, significantly outperforming the best alternative. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_15692 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement Nam, Jaehyun Yoon, Jinsung Chen, Jiefeng Shin, Jinwoo Arık, Sercan Ö. Pfister, Tomas Machine Learning Agents based on large language models (LLMs) for machine learning engineering (MLE) can automatically implement ML models via code generation. However, existing approaches to build such agents often rely heavily on inherent LLM knowledge and employ coarse exploration strategies that modify the entire code structure at once. This limits their ability to select effective task-specific models and perform deep exploration within specific components, such as experimenting extensively with feature engineering options. To overcome these, we propose MLE-STAR, a novel approach to build MLE agents. MLE-STAR first leverages external knowledge by using a search engine to retrieve effective models from the web, forming an initial solution, then iteratively refines it by exploring various strategies targeting specific ML components. This exploration is guided by ablation studies analyzing the impact of individual code blocks. Furthermore, we introduce a novel ensembling method using an effective strategy suggested by MLE-STAR. Our experimental results show that MLE-STAR achieves medals in 64% of the Kaggle competitions on the MLE-bench Lite, significantly outperforming the best alternative. |
| title | MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.15692 |