EvoGrad: Metaheuristics in a Differentiable Wonderland
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913882626326528 |
|---|---|
| author | Citterio, Beatrice F. R. Tangherloni, Andrea |
| author_facet | Citterio, Beatrice F. R. Tangherloni, Andrea |
| contents | Differentiable programming has revolutionised optimisation by enabling efficient gradient-based training of complex models, such as Deep Neural Networks (NNs) with billions and trillions of parameters. However, traditional Evolutionary Computation (EC) and Swarm Intelligence (SI) algorithms, widely successful in discrete or complex search spaces, typically do not leverage local gradient information, limiting their optimisation efficiency. In this paper, we introduce EvoGrad, a unified differentiable framework that integrates EC and SI with gradient-based optimisation through backpropagation. EvoGrad converts conventional evolutionary and swarm operators (e.g., selection, mutation, crossover, and particle updates) into differentiable operators, facilitating end-to-end gradient optimisation. Extensive experiments on benchmark optimisation functions and training of small NN regressors reveal that our differentiable versions of EC and SI metaheuristics consistently outperform traditional, gradient-agnostic algorithms in most scenarios. Our results show the substantial benefits of fully differentiable evolutionary and swarm optimisation, setting a new standard for hybrid optimisation frameworks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_06320 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EvoGrad: Metaheuristics in a Differentiable Wonderland Citterio, Beatrice F. R. Tangherloni, Andrea Neural and Evolutionary Computing Differentiable programming has revolutionised optimisation by enabling efficient gradient-based training of complex models, such as Deep Neural Networks (NNs) with billions and trillions of parameters. However, traditional Evolutionary Computation (EC) and Swarm Intelligence (SI) algorithms, widely successful in discrete or complex search spaces, typically do not leverage local gradient information, limiting their optimisation efficiency. In this paper, we introduce EvoGrad, a unified differentiable framework that integrates EC and SI with gradient-based optimisation through backpropagation. EvoGrad converts conventional evolutionary and swarm operators (e.g., selection, mutation, crossover, and particle updates) into differentiable operators, facilitating end-to-end gradient optimisation. Extensive experiments on benchmark optimisation functions and training of small NN regressors reveal that our differentiable versions of EC and SI metaheuristics consistently outperform traditional, gradient-agnostic algorithms in most scenarios. Our results show the substantial benefits of fully differentiable evolutionary and swarm optimisation, setting a new standard for hybrid optimisation frameworks. |
| title | EvoGrad: Metaheuristics in a Differentiable Wonderland |
| topic | Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2506.06320 |