Evolution Strategies at the Hyperscale
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915800243240960 |
|---|---|
| author | Sarkar, Bidipta Fellows, Mattie Duque, Juan Agustin Letcher, Alistair Villares, Antonio León Sims, Anya Wibault, Clarisse Samsonov, Dmitry Cope, Dylan Liesen, Jarek Li, Kang Seier, Lukas Wolf, Theo Berdica, Uljad Mohl, Valentin Goldie, Alexander David Courville, Aaron Sevegnani, Karin Whiteson, Shimon Foerster, Jakob Nicolaus |
| author_facet | Sarkar, Bidipta Fellows, Mattie Duque, Juan Agustin Letcher, Alistair Villares, Antonio León Sims, Anya Wibault, Clarisse Samsonov, Dmitry Cope, Dylan Liesen, Jarek Li, Kang Seier, Lukas Wolf, Theo Berdica, Uljad Mohl, Valentin Goldie, Alexander David Courville, Aaron Sevegnani, Karin Whiteson, Shimon Foerster, Jakob Nicolaus |
| contents | Evolution Strategies (ES) is a class of powerful black-box optimisation methods that are highly parallelisable and can handle non-differentiable and noisy objectives. However, naïve ES becomes prohibitively expensive at scale on GPUs due to the low arithmetic intensity of batched matrix multiplications with unstructured random perturbations. We introduce Evolution Guided GeneRal Optimisation via Low-rank Learning (EGGROLL), which improves arithmetic intensity by structuring individual perturbations as rank-$r$ matrices, resulting in a hundredfold increase in training speed for billion-parameter models at large population sizes, achieving up to 91% of the throughput of pure batch inference. We provide a rigorous theoretical analysis of Gaussian ES for high-dimensional parameter objectives, investigating conditions needed for ES updates to converge in high dimensions. Our results reveal a linearising effect, and proving consistency between EGGROLL and ES as parameter dimension increases. Our experiments show that EGGROLL: (1) enables the stable pretraining of nonlinear recurrent language models that operate purely in integer datatypes, (2) is competitive with GRPO for post-training LLMs on reasoning tasks, and (3) does not compromise performance compared to ES in tabula rasa RL settings, despite being faster. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_16652 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evolution Strategies at the Hyperscale Sarkar, Bidipta Fellows, Mattie Duque, Juan Agustin Letcher, Alistair Villares, Antonio León Sims, Anya Wibault, Clarisse Samsonov, Dmitry Cope, Dylan Liesen, Jarek Li, Kang Seier, Lukas Wolf, Theo Berdica, Uljad Mohl, Valentin Goldie, Alexander David Courville, Aaron Sevegnani, Karin Whiteson, Shimon Foerster, Jakob Nicolaus Machine Learning Artificial Intelligence Evolution Strategies (ES) is a class of powerful black-box optimisation methods that are highly parallelisable and can handle non-differentiable and noisy objectives. However, naïve ES becomes prohibitively expensive at scale on GPUs due to the low arithmetic intensity of batched matrix multiplications with unstructured random perturbations. We introduce Evolution Guided GeneRal Optimisation via Low-rank Learning (EGGROLL), which improves arithmetic intensity by structuring individual perturbations as rank-$r$ matrices, resulting in a hundredfold increase in training speed for billion-parameter models at large population sizes, achieving up to 91% of the throughput of pure batch inference. We provide a rigorous theoretical analysis of Gaussian ES for high-dimensional parameter objectives, investigating conditions needed for ES updates to converge in high dimensions. Our results reveal a linearising effect, and proving consistency between EGGROLL and ES as parameter dimension increases. Our experiments show that EGGROLL: (1) enables the stable pretraining of nonlinear recurrent language models that operate purely in integer datatypes, (2) is competitive with GRPO for post-training LLMs on reasoning tasks, and (3) does not compromise performance compared to ES in tabula rasa RL settings, despite being faster. |
| title | Evolution Strategies at the Hyperscale |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2511.16652 |