Fully Adaptive Zeroth-Order Method for Minimizing Functions with Compressible Gradients
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915389568450560 |
|---|---|
| author | Grapiglia, Geovani Nunes McKenzie, Daniel |
| author_facet | Grapiglia, Geovani Nunes McKenzie, Daniel |
| contents | We propose an adaptive zeroth-order method for minimizing differentiable functions with $L$-Lipschitz continuous gradients. The method is designed to take advantage of the eventual compressibility of the gradient of the objective function, but it does not require knowledge of the approximate sparsity level $s$ or the Lipschitz constant $L$ of the gradient. We show that the new method performs no more than $O\left(n^{2}ε^{-2}\right)$ function evaluations to find an $ε$-approximate stationary point of an objective function with $n$ variables. Assuming additionally that the gradients of the objective function are compressible, we obtain an improved complexity bound of $O\left(s\log\left(n\right)ε^{-2}\right)$ function evaluations, which holds with high probability. Preliminary numerical results illustrate the efficiency of the proposed method and demonstrate that it can significantly outperform its non-adaptive counterpart. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_11616 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Fully Adaptive Zeroth-Order Method for Minimizing Functions with Compressible Gradients Grapiglia, Geovani Nunes McKenzie, Daniel Optimization and Control We propose an adaptive zeroth-order method for minimizing differentiable functions with $L$-Lipschitz continuous gradients. The method is designed to take advantage of the eventual compressibility of the gradient of the objective function, but it does not require knowledge of the approximate sparsity level $s$ or the Lipschitz constant $L$ of the gradient. We show that the new method performs no more than $O\left(n^{2}ε^{-2}\right)$ function evaluations to find an $ε$-approximate stationary point of an objective function with $n$ variables. Assuming additionally that the gradients of the objective function are compressible, we obtain an improved complexity bound of $O\left(s\log\left(n\right)ε^{-2}\right)$ function evaluations, which holds with high probability. Preliminary numerical results illustrate the efficiency of the proposed method and demonstrate that it can significantly outperform its non-adaptive counterpart. |
| title | Fully Adaptive Zeroth-Order Method for Minimizing Functions with Compressible Gradients |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2501.11616 |