Gradient Projection onto Historical Descent Directions for Communication-Efficient Federated Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914143363137536 |
|---|---|
| author | Descours, Arnaud Deroose, Léonard Ramon, Jan |
| author_facet | Descours, Arnaud Deroose, Léonard Ramon, Jan |
| contents | Federated Learning (FL) enables decentralized model training across multiple clients while optionally preserving data privacy. However, communication efficiency remains a critical bottleneck, particularly for large-scale models. In this work, we introduce two complementary algorithms: ProjFL, designed for unbiased compressors, and ProjFL+EF, tailored for biased compressors through an Error Feedback mechanism. Both methods rely on projecting local gradients onto a shared client-server subspace spanned by historical descent directions, enabling efficient information exchange with minimal communication overhead. We establish convergence guarantees for both algorithms under strongly convex, convex, and non-convex settings. Empirical evaluations on standard FL classification benchmarks with deep neural networks show that ProjFL and ProjFL+EF achieve accuracy comparable to existing baselines while substantially reducing communication costs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_05593 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Gradient Projection onto Historical Descent Directions for Communication-Efficient Federated Learning Descours, Arnaud Deroose, Léonard Ramon, Jan Machine Learning Neural and Evolutionary Computing Optimization and Control Statistics Theory Federated Learning (FL) enables decentralized model training across multiple clients while optionally preserving data privacy. However, communication efficiency remains a critical bottleneck, particularly for large-scale models. In this work, we introduce two complementary algorithms: ProjFL, designed for unbiased compressors, and ProjFL+EF, tailored for biased compressors through an Error Feedback mechanism. Both methods rely on projecting local gradients onto a shared client-server subspace spanned by historical descent directions, enabling efficient information exchange with minimal communication overhead. We establish convergence guarantees for both algorithms under strongly convex, convex, and non-convex settings. Empirical evaluations on standard FL classification benchmarks with deep neural networks show that ProjFL and ProjFL+EF achieve accuracy comparable to existing baselines while substantially reducing communication costs. |
| title | Gradient Projection onto Historical Descent Directions for Communication-Efficient Federated Learning |
| topic | Machine Learning Neural and Evolutionary Computing Optimization and Control Statistics Theory |
| url | https://arxiv.org/abs/2511.05593 |