Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916233668984832 |
|---|---|
| author | Reifenstein, Sam Leleu, Timothee Yamamoto, Yoshihisa |
| author_facet | Reifenstein, Sam Leleu, Timothee Yamamoto, Yoshihisa |
| contents | We propose a novel algorithm that extends the methods of ball smoothing and Gaussian smoothing for noisy derivative-free optimization by accounting for the heterogeneous curvature of the objective function. The algorithm dynamically adapts the shape of the smoothing kernel to approximate the Hessian of the objective function around a local optimum. This approach significantly reduces the error in estimating the gradient from noisy evaluations through sampling. We demonstrate the efficacy of our method through numerical experiments on artificial problems. Additionally, we show improved performance when tuning NP-hard combinatorial optimization solvers compared to existing state-of-the-art heuristic derivative-free and Bayesian optimization methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_01731 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization Reifenstein, Sam Leleu, Timothee Yamamoto, Yoshihisa Machine Learning Optimization and Control We propose a novel algorithm that extends the methods of ball smoothing and Gaussian smoothing for noisy derivative-free optimization by accounting for the heterogeneous curvature of the objective function. The algorithm dynamically adapts the shape of the smoothing kernel to approximate the Hessian of the objective function around a local optimum. This approach significantly reduces the error in estimating the gradient from noisy evaluations through sampling. We demonstrate the efficacy of our method through numerical experiments on artificial problems. Additionally, we show improved performance when tuning NP-hard combinatorial optimization solvers compared to existing state-of-the-art heuristic derivative-free and Bayesian optimization methods. |
| title | Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2405.01731 |