Heterogeneous-Horizon Exact-Weight Local SGD
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914512511172608 |
|---|---|
| author | Pasechnyuk-Vilensky, Dmitry Takáč, Martin |
| author_facet | Pasechnyuk-Vilensky, Dmitry Takáč, Martin |
| contents | We study adaptive aggregation for heterogeneous local SGD in convex finite-sum optimization, allowing heterogeneous local horizons, minibatch sizes, gradient noise, and participation. We introduce HEW-Local SGD, a corrected local-SGD method that chooses nodewise server weights by minimizing an explicit one-round upper bound on the next objective value. This yields an exact local-control formulation with a threshold simplex update, separable amplitude updates, and a one-step guarantee under arbitrary predictable participation. We also introduce two post-local variants: a corrected heterogeneous method and a simpler homogeneous specialization. We establish one-step guarantees and global benchmark-style convergence results. In the regimes where comparison is appropriate, the theory matches the qualitative communication-efficient picture of recent LocalSGD/SCAFFOLD analyses, while also giving explicit guarantees for unequal local horizons. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_24463 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Heterogeneous-Horizon Exact-Weight Local SGD Pasechnyuk-Vilensky, Dmitry Takáč, Martin Optimization and Control 65K10 G.1.6 We study adaptive aggregation for heterogeneous local SGD in convex finite-sum optimization, allowing heterogeneous local horizons, minibatch sizes, gradient noise, and participation. We introduce HEW-Local SGD, a corrected local-SGD method that chooses nodewise server weights by minimizing an explicit one-round upper bound on the next objective value. This yields an exact local-control formulation with a threshold simplex update, separable amplitude updates, and a one-step guarantee under arbitrary predictable participation. We also introduce two post-local variants: a corrected heterogeneous method and a simpler homogeneous specialization. We establish one-step guarantees and global benchmark-style convergence results. In the regimes where comparison is appropriate, the theory matches the qualitative communication-efficient picture of recent LocalSGD/SCAFFOLD analyses, while also giving explicit guarantees for unequal local horizons. |
| title | Heterogeneous-Horizon Exact-Weight Local SGD |
| topic | Optimization and Control 65K10 G.1.6 |
| url | https://arxiv.org/abs/2604.24463 |