Last Iterate Convergence of AdaGrad-Norm for Convex Non-Smooth Optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911586777563136 |
|---|---|
| author | Preobrazhenskaia, Margarita Sidorov, Makar Preobrazhenskii, Igor Gorbunov, Eduard |
| author_facet | Preobrazhenskaia, Margarita Sidorov, Makar Preobrazhenskii, Igor Gorbunov, Eduard |
| contents | We study the convergence of the last iterate (i.e., the $(N+1)$-th iterate) of the AdaGrad method. Although AdaGrad -- an adaptive subgradient method -- underpins a wide class of algorithms, most existing convergence analyses focus on averaged (or best) iterates. We derive worst-case upper bounds on the suboptimality of the final point and show that, with an optimally tuned stepsize parameter, the last iterate converges at the rate $O(1/N^{1/4})$. We complement this guarantee with matching lower-bound constructions, proving that this rate is tight and that AdaGrad's last-iterate rate is strictly worse than the classical $O(1/N^{1/2})$ rate for its averaged iterate. Technically, our analysis introduces an exponent parameter that captures the growth of the cumulative squared subgradients; combined with the last-iterate inequality of Zamani and Glineur (2025), this reduces the problem to bounding a particular series. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_10728 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Last Iterate Convergence of AdaGrad-Norm for Convex Non-Smooth Optimization Preobrazhenskaia, Margarita Sidorov, Makar Preobrazhenskii, Igor Gorbunov, Eduard Optimization and Control We study the convergence of the last iterate (i.e., the $(N+1)$-th iterate) of the AdaGrad method. Although AdaGrad -- an adaptive subgradient method -- underpins a wide class of algorithms, most existing convergence analyses focus on averaged (or best) iterates. We derive worst-case upper bounds on the suboptimality of the final point and show that, with an optimally tuned stepsize parameter, the last iterate converges at the rate $O(1/N^{1/4})$. We complement this guarantee with matching lower-bound constructions, proving that this rate is tight and that AdaGrad's last-iterate rate is strictly worse than the classical $O(1/N^{1/2})$ rate for its averaged iterate. Technically, our analysis introduces an exponent parameter that captures the growth of the cumulative squared subgradients; combined with the last-iterate inequality of Zamani and Glineur (2025), this reduces the problem to bounding a particular series. |
| title | Last Iterate Convergence of AdaGrad-Norm for Convex Non-Smooth Optimization |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2604.10728 |