Polyak Stepsize: Estimating Optimal Functional Values Without Parameters or Prior Knowledge
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908500154646528 |
|---|---|
| author | Abdukhakimov, Farshed Pham, Cuong Anh Horváth, Samuel Takáč, Martin Hanzely, Slavomır |
| author_facet | Abdukhakimov, Farshed Pham, Cuong Anh Horváth, Samuel Takáč, Martin Hanzely, Slavomır |
| contents | The Polyak stepsize for Gradient Descent is known for its fast convergence but requires prior knowledge of the optimal functional value, which is often unavailable in practice. In this paper, we propose a parameter-free approach that estimates this unknown value during the algorithm's execution, enabling a parameter-free stepsize schedule. Our method maintains two sequences of iterates: one with a higher functional value is updated using the Polyak stepsize, and the other one with a lower functional value is used as an estimate of the optimal functional value. We provide a theoretical analysis of the approach and validate its performance through numerical experiments. The results demonstrate that our method achieves competitive performance without relying on prior function-dependent information. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_17288 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Polyak Stepsize: Estimating Optimal Functional Values Without Parameters or Prior Knowledge Abdukhakimov, Farshed Pham, Cuong Anh Horváth, Samuel Takáč, Martin Hanzely, Slavomır Optimization and Control The Polyak stepsize for Gradient Descent is known for its fast convergence but requires prior knowledge of the optimal functional value, which is often unavailable in practice. In this paper, we propose a parameter-free approach that estimates this unknown value during the algorithm's execution, enabling a parameter-free stepsize schedule. Our method maintains two sequences of iterates: one with a higher functional value is updated using the Polyak stepsize, and the other one with a lower functional value is used as an estimate of the optimal functional value. We provide a theoretical analysis of the approach and validate its performance through numerical experiments. The results demonstrate that our method achieves competitive performance without relying on prior function-dependent information. |
| title | Polyak Stepsize: Estimating Optimal Functional Values Without Parameters or Prior Knowledge |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2508.17288 |