Cover Image

MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chan, Jun Shern, Chowdhury, Neil, Jaffe, Oliver, Aung, James, Sherburn, Dane, Mays, Evan, Starace, Giulio, Liu, Kevin, Maksin, Leon, Patwardhan, Tejal, Weng, Lilian, Mądry, Aleksander
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PaperBench: Evaluating AI's Ability to Replicate AI Research
by: Starace, Giulio, et al.
Published: (2025)

AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench
by: Toledo, Edan, et al.
Published: (2025)

MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering
by: Qiang, Rushi, et al.
Published: (2025)

MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement
by: Nam, Jaehyun, et al.
Published: (2025)

Can Language Models Explain Their Own Classification Behavior?
by: Sherburn, Dane, et al.
Published: (2024)

SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?
by: Miserendino, Samuel, et al.
Published: (2025)

Learning to Attribute with Attention
by: Cohen-Wang, Benjamin, et al.
Published: (2025)

FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks
by: Wang, Miles, et al.
Published: (2026)

FML-bench: Benchmarking Machine Learning Agents for Scientific Research
by: Zou, Qiran, et al.
Published: (2025)

Attribute-to-Delete: Machine Unlearning via Datamodel Matching
by: Georgiev, Kristian, et al.
Published: (2024)

Small-to-Large Generalization: Data Influences Models Consistently Across Scale
by: Khaddaj, Alaa, et al.
Published: (2025)

DsDm: Model-Aware Dataset Selection with Datamodels
by: Engstrom, Logan, et al.
Published: (2024)

Decomposing and Editing Predictions by Modeling Model Computation
by: Shah, Harshay, et al.
Published: (2024)

Accelerating Transpilation in Quantum Machine Learning with Haiqu's Rivet-transpiler
by: Kaczmarek, Aleksander, et al.
Published: (2025)

Ask Your Distribution Shift if Pre-Training is Right for You
by: Cohen-Wang, Benjamin, et al.
Published: (2024)

User Strategization and Trustworthy Algorithms
by: Cen, Sarah H., et al.
Published: (2023)

Beyond MLE: Investigating SEARNN for Low-Resourced Neural Machine Translation
by: Emezue, Chris
Published: (2024)

Python workflow for segmenting multiphase flow in porous rocks
by: Spurin, Catherine, et al.
Published: (2024)

Adversarial Machine Learning: Attacking and Safeguarding Image Datasets
by: Chowdhury, Koushik
Published: (2025)

BudgetMLAgent: A Cost-Effective LLM Multi-Agent system for Automating Machine Learning Tasks
by: Gandhi, Shubham, et al.
Published: (2024)

Risk Prediction of Cardiovascular Disease for Diabetic Patients with Machine Learning and Deep Learning Techniques
by: Chowdhury, Esha
Published: (2025)

Do Large Language Model Benchmarks Test Reliability?
by: Vendrow, Joshua, et al.
Published: (2025)

DataMIL: Selecting Data for Robot Imitation Learning with Datamodels
by: Dass, Shivin, et al.
Published: (2025)

A General Approach for Determining Applicability Domain of Machine Learning Models
by: Schultz, Lane E., et al.
Published: (2024)

Legal Zero-Days: A Novel Risk Vector for Advanced AI Systems
by: Sadler, Greg, et al.
Published: (2025)

Unlocking Potential Catalysts: A Machine Learning Approach with Bayesian and Regression Models
by: Chandra Chowdhury
Published: (2024)

How Can Machine Learning Accelerate CALPHAD Free Energy Modeling?
by: Shen, Chen, et al.
Published: (2026)

ContextCite: Attributing Model Generation to Context
by: Cohen-Wang, Benjamin, et al.
Published: (2024)

Learning to Ideate for Machine Learning Engineering Agents
by: Zhang, Yunxiang, et al.
Published: (2026)

Reinforcement Learning for Machine Learning Engineering Agents
by: Yang, Sherry, et al.
Published: (2025)

Valid Inference for Machine Learning Model Parameters
by: Dey, Neil, et al.
Published: (2023)

Is Agentic AI Ready for Real-World Hardware Engineering? A Deep Dive with Phoenix-bench
by: Zou, Qingyun, et al.
Published: (2026)

Differentially Private and Adversarially Robust Machine Learning: An Empirical Evaluation
by: Thakkar, Janvi, et al.
Published: (2024)

MLE-Smith: Scaling MLE Tasks with Automated Multi-Agent Pipeline
by: Qiang, Rushi, et al.
Published: (2025)

Proximal Reliability Optimization for Reinforcement Learning
by: Patwardhan, Narendra, et al.
Published: (2019)

CL-bench: A Benchmark for Context Learning
by: Dou, Shihan, et al.
Published: (2026)

Bootstrapping string models with entanglement minimization and Machine-Learning
by: Bhat, Faizan, et al.
Published: (2024)

Performance Analysis of Machine Learning Algorithms in Chronic Kidney Disease Prediction
by: Ahmed, Iftekhar, et al.
Published: (2025)

Etrasimod for alopecia areata: The scenario for a less extensive and moderate form
by: M. Starace
Published: (2025)

Optimising for Energy Efficiency and Performance in Machine Learning
by: Ferreira, Emile Dos Santos, et al.
Published: (2026)