Procedural Generation of Algorithm Discovery Tasks in Machine Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911526488637440 |
|---|---|
| author | Goldie, Alexander D. Wang, Zilin Hayler, Adrian Nathani, Deepak Toledo, Edan Thampiratwong, Ken Kalisz, Aleksandra Beukman, Michael Letcher, Alistair Reddy, Shashank Wibault, Clarisse Wolf, Theo O'Neill, Charles Berdica, Uljad Roberts, Nicholas Rahmani, Saeed Erlebach, Hannah Raileanu, Roberta Whiteson, Shimon Foerster, Jakob N. |
| author_facet | Goldie, Alexander D. Wang, Zilin Hayler, Adrian Nathani, Deepak Toledo, Edan Thampiratwong, Ken Kalisz, Aleksandra Beukman, Michael Letcher, Alistair Reddy, Shashank Wibault, Clarisse Wolf, Theo O'Neill, Charles Berdica, Uljad Roberts, Nicholas Rahmani, Saeed Erlebach, Hannah Raileanu, Roberta Whiteson, Shimon Foerster, Jakob N. |
| contents | Automating the development of machine learning algorithms has the potential to unlock new breakthroughs. However, our ability to improve and evaluate algorithm discovery systems has thus far been limited by existing task suites. They suffer from many issues, such as: poor evaluation methodologies; data contamination; and containing saturated or very similar problems. Here, we introduce DiscoGen, a procedural generator of algorithm discovery tasks for machine learning, such as developing optimisers for reinforcement learning or loss functions for image classification. Motivated by the success of procedural generation in reinforcement learning, DiscoGen spans millions of tasks of varying difficulty and complexity from a range of machine learning fields. These tasks are specified by a small number of configuration parameters and can be used to optimise algorithm discovery agents (ADAs). We present DiscoBench, a benchmark consisting of a fixed, small subset of DiscoGen tasks for principled evaluation of ADAs. Finally, we propose a number of ambitious, impactful research directions enabled by DiscoGen, in addition to experiments demonstrating its use for prompt optimisation of an ADA. DiscoGen is released open-source at https://github.com/AlexGoldie/discogen. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_17863 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Procedural Generation of Algorithm Discovery Tasks in Machine Learning Goldie, Alexander D. Wang, Zilin Hayler, Adrian Nathani, Deepak Toledo, Edan Thampiratwong, Ken Kalisz, Aleksandra Beukman, Michael Letcher, Alistair Reddy, Shashank Wibault, Clarisse Wolf, Theo O'Neill, Charles Berdica, Uljad Roberts, Nicholas Rahmani, Saeed Erlebach, Hannah Raileanu, Roberta Whiteson, Shimon Foerster, Jakob N. Machine Learning Artificial Intelligence Automating the development of machine learning algorithms has the potential to unlock new breakthroughs. However, our ability to improve and evaluate algorithm discovery systems has thus far been limited by existing task suites. They suffer from many issues, such as: poor evaluation methodologies; data contamination; and containing saturated or very similar problems. Here, we introduce DiscoGen, a procedural generator of algorithm discovery tasks for machine learning, such as developing optimisers for reinforcement learning or loss functions for image classification. Motivated by the success of procedural generation in reinforcement learning, DiscoGen spans millions of tasks of varying difficulty and complexity from a range of machine learning fields. These tasks are specified by a small number of configuration parameters and can be used to optimise algorithm discovery agents (ADAs). We present DiscoBench, a benchmark consisting of a fixed, small subset of DiscoGen tasks for principled evaluation of ADAs. Finally, we propose a number of ambitious, impactful research directions enabled by DiscoGen, in addition to experiments demonstrating its use for prompt optimisation of an ADA. DiscoGen is released open-source at https://github.com/AlexGoldie/discogen. |
| title | Procedural Generation of Algorithm Discovery Tasks in Machine Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2603.17863 |