Local Steps Speed Up Local GD for Heterogeneous Distributed Logistic Regression
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910930710822912 |
|---|---|
| author | Crawshaw, Michael Woodworth, Blake Liu, Mingrui |
| author_facet | Crawshaw, Michael Woodworth, Blake Liu, Mingrui |
| contents | We analyze two variants of Local Gradient Descent applied to distributed logistic regression with heterogeneous, separable data and show convergence at the rate $O(1/KR)$ for $K$ local steps and sufficiently large $R$ communication rounds. In contrast, all existing convergence guarantees for Local GD applied to any problem are at least $Ω(1/R)$, meaning they fail to show the benefit of local updates. The key to our improved guarantee is showing progress on the logistic regression objective when using a large stepsize $η\gg 1/K$, whereas prior analysis depends on $η\leq 1/K$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_13790 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Local Steps Speed Up Local GD for Heterogeneous Distributed Logistic Regression Crawshaw, Michael Woodworth, Blake Liu, Mingrui Machine Learning We analyze two variants of Local Gradient Descent applied to distributed logistic regression with heterogeneous, separable data and show convergence at the rate $O(1/KR)$ for $K$ local steps and sufficiently large $R$ communication rounds. In contrast, all existing convergence guarantees for Local GD applied to any problem are at least $Ω(1/R)$, meaning they fail to show the benefit of local updates. The key to our improved guarantee is showing progress on the logistic regression objective when using a large stepsize $η\gg 1/K$, whereas prior analysis depends on $η\leq 1/K$. |
| title | Local Steps Speed Up Local GD for Heterogeneous Distributed Logistic Regression |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2501.13790 |