On the boundedness of the sequence generated by minibatch stochastic gradient descent
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913917573267456 |
|---|---|
| author | Bauschke, Heinz H. Tung, Tran Thanh |
| author_facet | Bauschke, Heinz H. Tung, Tran Thanh |
| contents | Stochastic Gradient Descent (SGD) with Polyak's stepsize has recently gained renewed attention in stochastic optimization. Recently, Orvieto, Lacoste-Julien, and Loizou introduced a decreasing variant of Polyak's stepsize, where convergence relies on a boundedness assumption of the iterates. They established that this assumption holds under strong convexity. In this paper, we extend their result by proving that boundedness also holds for a broader class of objective functions, including coercive functions. We also present a case in which boundedness may or may not hold. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_23303 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the boundedness of the sequence generated by minibatch stochastic gradient descent Bauschke, Heinz H. Tung, Tran Thanh Optimization and Control Functional Analysis Primary 90C15, 90C25, 65K05, Secondary 68T07, 68W20, 68W40 Stochastic Gradient Descent (SGD) with Polyak's stepsize has recently gained renewed attention in stochastic optimization. Recently, Orvieto, Lacoste-Julien, and Loizou introduced a decreasing variant of Polyak's stepsize, where convergence relies on a boundedness assumption of the iterates. They established that this assumption holds under strong convexity. In this paper, we extend their result by proving that boundedness also holds for a broader class of objective functions, including coercive functions. We also present a case in which boundedness may or may not hold. |
| title | On the boundedness of the sequence generated by minibatch stochastic gradient descent |
| topic | Optimization and Control Functional Analysis Primary 90C15, 90C25, 65K05, Secondary 68T07, 68W20, 68W40 |
| url | https://arxiv.org/abs/2506.23303 |