Stochastic Subgradient Methods with Guaranteed Global Stability in Nonsmooth Nonconvex Optimization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913542291062784 |
|---|---|
| author | Xiao, Nachuan Hu, Xiaoyin Toh, Kim-Chuan |
| author_facet | Xiao, Nachuan Hu, Xiaoyin Toh, Kim-Chuan |
| contents | In this paper, we focus on providing convergence guarantees for stochastic subgradient methods in minimizing nonsmooth nonconvex functions. We first investigate the global stability of a general framework for stochastic subgradient methods, where the corresponding differential inclusion admits a coercive Lyapunov function. We prove that, for any sequence of sufficiently small stepsizes and approximation parameters, coupled with sufficiently controlled noises, the iterates are uniformly bounded and asymptotically stabilize around the stable set of its corresponding differential inclusion. Moreover, we develop an improved analysis to apply our proposed framework to establish the global stability of a wide range of stochastic subgradient methods, where the corresponding Lyapunov functions are possibly non-coercive. These theoretical results illustrate the promising potential of our proposed framework for establishing the global stability of various stochastic subgradient methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_10053 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Stochastic Subgradient Methods with Guaranteed Global Stability in Nonsmooth Nonconvex Optimization Xiao, Nachuan Hu, Xiaoyin Toh, Kim-Chuan Optimization and Control Artificial Intelligence Machine Learning In this paper, we focus on providing convergence guarantees for stochastic subgradient methods in minimizing nonsmooth nonconvex functions. We first investigate the global stability of a general framework for stochastic subgradient methods, where the corresponding differential inclusion admits a coercive Lyapunov function. We prove that, for any sequence of sufficiently small stepsizes and approximation parameters, coupled with sufficiently controlled noises, the iterates are uniformly bounded and asymptotically stabilize around the stable set of its corresponding differential inclusion. Moreover, we develop an improved analysis to apply our proposed framework to establish the global stability of a wide range of stochastic subgradient methods, where the corresponding Lyapunov functions are possibly non-coercive. These theoretical results illustrate the promising potential of our proposed framework for establishing the global stability of various stochastic subgradient methods. |
| title | Stochastic Subgradient Methods with Guaranteed Global Stability in Nonsmooth Nonconvex Optimization |
| topic | Optimization and Control Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2307.10053 |