Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2311.03154 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929337401344000 |
|---|---|
| author | Li, Yipeng Lyu, Xinchen |
| author_facet | Li, Yipeng Lyu, Xinchen |
| contents | There are two categories of methods in Federated Learning (FL) for joint training across multiple clients: i) parallel FL (PFL), where clients train models in a parallel manner; and ii) sequential FL (SFL), where clients train models in a sequential manner. In contrast to that of PFL, the convergence theory of SFL on heterogeneous data is still lacking. In this paper, we establish the convergence guarantees of SFL for strongly/general/non-convex objectives on heterogeneous data. The convergence guarantees of SFL are better than that of PFL on heterogeneous data with both full and partial client participation. Experimental results validate the counterintuitive analysis result that SFL outperforms PFL on extremely heterogeneous data in cross-device settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_03154 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Convergence Analysis of Sequential Federated Learning on Heterogeneous Data Li, Yipeng Lyu, Xinchen Machine Learning There are two categories of methods in Federated Learning (FL) for joint training across multiple clients: i) parallel FL (PFL), where clients train models in a parallel manner; and ii) sequential FL (SFL), where clients train models in a sequential manner. In contrast to that of PFL, the convergence theory of SFL on heterogeneous data is still lacking. In this paper, we establish the convergence guarantees of SFL for strongly/general/non-convex objectives on heterogeneous data. The convergence guarantees of SFL are better than that of PFL on heterogeneous data with both full and partial client participation. Experimental results validate the counterintuitive analysis result that SFL outperforms PFL on extremely heterogeneous data in cross-device settings. |
| title | Convergence Analysis of Sequential Federated Learning on Heterogeneous Data |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2311.03154 |