Almost Sure Convergence of Stochastic Approximation: An Interplay of Noise and Step Size
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914339506618368 |
|---|---|
| author | Nguyen, Quang Dinh Thien Nguyen, Duc Anh Nguyen, Hoang Huy Maguluri, Siva Theja |
| author_facet | Nguyen, Quang Dinh Thien Nguyen, Duc Anh Nguyen, Hoang Huy Maguluri, Siva Theja |
| contents | We study the almost sure convergence of the Stochastic Approximation algorithm to the fixed point $x^\star$ of a nonlinear operator under a negative drift condition and a general noise sequence with finite $p$-th moment for some $p > 1$. Classical almost sure convergence results of Stochastic Approximation are mostly analyzed for the square-integrable noise setting, and it is shown that any non-summable but square-summable step size sequence is sufficient to obtain almost sure convergence. However, such a limitation prevents wider algorithmic application. In particular, many applications in Machine Learning and Operations Research admit heavy-tailed noise with infinite variance, rendering such guarantees inapplicable. On the other hand, when a stronger condition on the noise is available, such guarantees on the step size would be too conservative, as practitioners would like to pick a larger step size for a more preferable convergence behavior. To this end, we show that any non-summable but $p$-th power summable step size sequence is sufficient to guarantee almost sure convergence, covering the gap in the literature.
Our guarantees are obtained using a universal Lyapunov drift argument. For the regime $p \in (1, 2)$, we show that using the Lyapunov function $\norm{x-x^\star}^p$ and applying a Taylor-like bound suffice. For $p > 2$, such an approach is no longer applicable, and therefore, we introduce a novel iterate projection technique to control the nonlinear terms produced by high-moment bounds and multiplicative noise. We believe our proof techniques and their implications could be of independent interest and pave the way for finite-time analysis of Stochastic Approximation under a general noise condition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_10580 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Almost Sure Convergence of Stochastic Approximation: An Interplay of Noise and Step Size Nguyen, Quang Dinh Thien Nguyen, Duc Anh Nguyen, Hoang Huy Maguluri, Siva Theja Optimization and Control Probability We study the almost sure convergence of the Stochastic Approximation algorithm to the fixed point $x^\star$ of a nonlinear operator under a negative drift condition and a general noise sequence with finite $p$-th moment for some $p > 1$. Classical almost sure convergence results of Stochastic Approximation are mostly analyzed for the square-integrable noise setting, and it is shown that any non-summable but square-summable step size sequence is sufficient to obtain almost sure convergence. However, such a limitation prevents wider algorithmic application. In particular, many applications in Machine Learning and Operations Research admit heavy-tailed noise with infinite variance, rendering such guarantees inapplicable. On the other hand, when a stronger condition on the noise is available, such guarantees on the step size would be too conservative, as practitioners would like to pick a larger step size for a more preferable convergence behavior. To this end, we show that any non-summable but $p$-th power summable step size sequence is sufficient to guarantee almost sure convergence, covering the gap in the literature. Our guarantees are obtained using a universal Lyapunov drift argument. For the regime $p \in (1, 2)$, we show that using the Lyapunov function $\norm{x-x^\star}^p$ and applying a Taylor-like bound suffice. For $p > 2$, such an approach is no longer applicable, and therefore, we introduce a novel iterate projection technique to control the nonlinear terms produced by high-moment bounds and multiplicative noise. We believe our proof techniques and their implications could be of independent interest and pave the way for finite-time analysis of Stochastic Approximation under a general noise condition. |
| title | Almost Sure Convergence of Stochastic Approximation: An Interplay of Noise and Step Size |
| topic | Optimization and Control Probability |
| url | https://arxiv.org/abs/2602.10580 |