SCAFFLSA: Taming Heterogeneity in Federated Linear Stochastic Approximation and TD Learning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909969937334272 |
|---|---|
| author | Mangold, Paul Samsonov, Sergey Labbi, Safwan Levin, Ilya Alami, Reda Naumov, Alexey Moulines, Eric |
| author_facet | Mangold, Paul Samsonov, Sergey Labbi, Safwan Levin, Ilya Alami, Reda Naumov, Alexey Moulines, Eric |
| contents | In this paper, we analyze the sample and communication complexity of the federated linear stochastic approximation (FedLSA) algorithm. We explicitly quantify the effects of local training with agent heterogeneity. We show that the communication complexity of FedLSA scales polynomially with the inverse of the desired accuracy $ε$. To overcome this, we propose SCAFFLSA a new variant of FedLSA that uses control variates to correct for client drift, and establish its sample and communication complexities. We show that for statistically heterogeneous agents, its communication complexity scales logarithmically with the desired accuracy, similar to Scaffnew. An important finding is that, compared to the existing results for Scaffnew, the sample complexity scales with the inverse of the number of agents, a property referred to as linear speed-up. Achieving this linear speed-up requires completely new theoretical arguments. We apply the proposed method to federated temporal difference learning with linear function approximation and analyze the corresponding complexity improvements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_04114 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SCAFFLSA: Taming Heterogeneity in Federated Linear Stochastic Approximation and TD Learning Mangold, Paul Samsonov, Sergey Labbi, Safwan Levin, Ilya Alami, Reda Naumov, Alexey Moulines, Eric Machine Learning Optimization and Control In this paper, we analyze the sample and communication complexity of the federated linear stochastic approximation (FedLSA) algorithm. We explicitly quantify the effects of local training with agent heterogeneity. We show that the communication complexity of FedLSA scales polynomially with the inverse of the desired accuracy $ε$. To overcome this, we propose SCAFFLSA a new variant of FedLSA that uses control variates to correct for client drift, and establish its sample and communication complexities. We show that for statistically heterogeneous agents, its communication complexity scales logarithmically with the desired accuracy, similar to Scaffnew. An important finding is that, compared to the existing results for Scaffnew, the sample complexity scales with the inverse of the number of agents, a property referred to as linear speed-up. Achieving this linear speed-up requires completely new theoretical arguments. We apply the proposed method to federated temporal difference learning with linear function approximation and analyze the corresponding complexity improvements. |
| title | SCAFFLSA: Taming Heterogeneity in Federated Linear Stochastic Approximation and TD Learning |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2402.04114 |