Natural Evolutionary Search meets Probabilistic Numerics

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Osselin, Pierre, Adachi, Masaki, Dong, Xiaowen, Osborne, Michael A.
Format:	Preprint
Veröffentlicht:	2025
Schlagworte:	Machine Learning Neural and Evolutionary Computing
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

_version_	1866913935754526720
author	Osselin, Pierre Adachi, Masaki Dong, Xiaowen Osborne, Michael A.
author_facet	Osselin, Pierre Adachi, Masaki Dong, Xiaowen Osborne, Michael A.
contents	Zeroth-order local optimisation algorithms are essential for solving real-valued black-box optimisation problems. Among these, Natural Evolution Strategies (NES) represent a prominent class, particularly well-suited for scenarios where prior distributions are available. By optimising the objective function in the space of search distributions, NES algorithms naturally integrate prior knowledge during initialisation, making them effective in settings such as semi-supervised learning and user-prior belief frameworks. However, due to their reliance on random sampling and Monte Carlo estimates, NES algorithms can suffer from limited sample efficiency. In this paper, we introduce a novel class of algorithms, termed Probabilistic Natural Evolutionary Strategy Algorithms (ProbNES), which enhance the NES framework with Bayesian quadrature. We show that ProbNES algorithms consistently outperforms their non-probabilistic counterparts as well as global sample efficient methods such as Bayesian Optimisation (BO) or $π$BO across a wide range of tasks, including benchmark test functions, data-driven optimisation tasks, user-informed hyperparameter tuning tasks and locomotion tasks.
format	Preprint
id	arxiv_https___arxiv_org_abs_2507_07288
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Natural Evolutionary Search meets Probabilistic Numerics Osselin, Pierre Adachi, Masaki Dong, Xiaowen Osborne, Michael A. Machine Learning Neural and Evolutionary Computing Zeroth-order local optimisation algorithms are essential for solving real-valued black-box optimisation problems. Among these, Natural Evolution Strategies (NES) represent a prominent class, particularly well-suited for scenarios where prior distributions are available. By optimising the objective function in the space of search distributions, NES algorithms naturally integrate prior knowledge during initialisation, making them effective in settings such as semi-supervised learning and user-prior belief frameworks. However, due to their reliance on random sampling and Monte Carlo estimates, NES algorithms can suffer from limited sample efficiency. In this paper, we introduce a novel class of algorithms, termed Probabilistic Natural Evolutionary Strategy Algorithms (ProbNES), which enhance the NES framework with Bayesian quadrature. We show that ProbNES algorithms consistently outperforms their non-probabilistic counterparts as well as global sample efficient methods such as Bayesian Optimisation (BO) or $π$BO across a wide range of tasks, including benchmark test functions, data-driven optimisation tasks, user-informed hyperparameter tuning tasks and locomotion tasks.
title	Natural Evolutionary Search meets Probabilistic Numerics
topic	Machine Learning Neural and Evolutionary Computing
url	https://arxiv.org/abs/2507.07288