Open Problem: Anytime Convergence Rate of Gradient Descent
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929392185245696 |
|---|---|
| author | Kornowski, Guy Shamir, Ohad |
| author_facet | Kornowski, Guy Shamir, Ohad |
| contents | Recent results show that vanilla gradient descent can be accelerated for smooth convex objectives, merely by changing the stepsize sequence. We show that this can lead to surprisingly large errors indefinitely, and therefore ask: Is there any stepsize schedule for gradient descent that accelerates the classic $\mathcal{O}(1/T)$ convergence rate, at \emph{any} stopping time $T$? |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_13888 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Open Problem: Anytime Convergence Rate of Gradient Descent Kornowski, Guy Shamir, Ohad Optimization and Control Machine Learning Recent results show that vanilla gradient descent can be accelerated for smooth convex objectives, merely by changing the stepsize sequence. We show that this can lead to surprisingly large errors indefinitely, and therefore ask: Is there any stepsize schedule for gradient descent that accelerates the classic $\mathcal{O}(1/T)$ convergence rate, at \emph{any} stopping time $T$? |
| title | Open Problem: Anytime Convergence Rate of Gradient Descent |
| topic | Optimization and Control Machine Learning |
| url | https://arxiv.org/abs/2406.13888 |