Natural Variational Annealing for Multimodal Optimization
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909965738835968 |
|---|---|
| author | LeMinh, Tâm Arbel, Julyan Möllenhoff, Thomas Khan, Mohammad Emtiyaz Forbes, Florence |
| author_facet | LeMinh, Tâm Arbel, Julyan Möllenhoff, Thomas Khan, Mohammad Emtiyaz Forbes, Florence |
| contents | We introduce a new multimodal optimization approach called Natural Variational Annealing (NVA) that combines the strengths of three foundational concepts to simultaneously search for multiple global and local modes of black-box nonconvex objectives. First, it implements a simultaneous search by using variational posteriors, such as, mixtures of Gaussians. Second, it applies annealing to gradually trade off exploration for exploitation. Finally, it learns the variational search distribution using natural-gradient learning where updates resemble well-known and easy-to-implement algorithms. The three concepts come together in NVA giving rise to new algorithms and also allowing us to incorporate "fitness shaping", a core concept from evolutionary algorithms. We assess the quality of search on simulations and compare them to methods using gradient descent and evolution strategies. We also provide an application to a real-world inverse problem in planetary science. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_04667 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Natural Variational Annealing for Multimodal Optimization LeMinh, Tâm Arbel, Julyan Möllenhoff, Thomas Khan, Mohammad Emtiyaz Forbes, Florence Machine Learning Computation We introduce a new multimodal optimization approach called Natural Variational Annealing (NVA) that combines the strengths of three foundational concepts to simultaneously search for multiple global and local modes of black-box nonconvex objectives. First, it implements a simultaneous search by using variational posteriors, such as, mixtures of Gaussians. Second, it applies annealing to gradually trade off exploration for exploitation. Finally, it learns the variational search distribution using natural-gradient learning where updates resemble well-known and easy-to-implement algorithms. The three concepts come together in NVA giving rise to new algorithms and also allowing us to incorporate "fitness shaping", a core concept from evolutionary algorithms. We assess the quality of search on simulations and compare them to methods using gradient descent and evolution strategies. We also provide an application to a real-world inverse problem in planetary science. |
| title | Natural Variational Annealing for Multimodal Optimization |
| topic | Machine Learning Computation |
| url | https://arxiv.org/abs/2501.04667 |