Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.14979 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918251778277376 |
|---|---|
| author | Wang, Qi Shanbhag, Uday V. Xie, Yue |
| author_facet | Wang, Qi Shanbhag, Uday V. Xie, Yue |
| contents | Most existing rate and complexity guarantees for stochastic gradient methods in $L$-smooth settings mandates that such sequences be non-adaptive, non-increasing, and upper bounded by $\tfrac{a}{L}$ for $a > 0$. This requires knowledge of $L$ and may preclude larger steps. Motivated by these shortcomings, we present an Armijo-enabled stochastic linesearch framework with standard stochastic zeroth- and first-order oracles. The resulting steplength sequence is non-monotone and requires neither knowledge of $L$ nor any other problem parameters. We then prove that the expected stationarity residual diminishes at a rate of $\mathcal{O}(1/\sqrt{K})$, where $K$ denotes the iteration budget. Furthermore, the resulting iteration and sample complexities for computing an $ε$-stationary point are $\mathcal{O}(ε^{-2})$ and $\mathcal{O}\left(ε^{-4}\right)$. The proposed method allows for a simple nonsmooth convex component in the objective, addressed through proximal gradient updates. Analogous guarantees are provided in the Polyak-Lojasiewicz (PL) setting and convex regimes. Preliminary numerical experiments are seen to be promising. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_14979 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Parameter-Free Stochastic LineseArch Method (SLAM) for Minimizing Expectation Residuals Wang, Qi Shanbhag, Uday V. Xie, Yue Optimization and Control Most existing rate and complexity guarantees for stochastic gradient methods in $L$-smooth settings mandates that such sequences be non-adaptive, non-increasing, and upper bounded by $\tfrac{a}{L}$ for $a > 0$. This requires knowledge of $L$ and may preclude larger steps. Motivated by these shortcomings, we present an Armijo-enabled stochastic linesearch framework with standard stochastic zeroth- and first-order oracles. The resulting steplength sequence is non-monotone and requires neither knowledge of $L$ nor any other problem parameters. We then prove that the expected stationarity residual diminishes at a rate of $\mathcal{O}(1/\sqrt{K})$, where $K$ denotes the iteration budget. Furthermore, the resulting iteration and sample complexities for computing an $ε$-stationary point are $\mathcal{O}(ε^{-2})$ and $\mathcal{O}\left(ε^{-4}\right)$. The proposed method allows for a simple nonsmooth convex component in the objective, addressed through proximal gradient updates. Analogous guarantees are provided in the Polyak-Lojasiewicz (PL) setting and convex regimes. Preliminary numerical experiments are seen to be promising. |
| title | A Parameter-Free Stochastic LineseArch Method (SLAM) for Minimizing Expectation Residuals |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2512.14979 |