Efficient Wireless Federated Learning via Low-Rank Gradient Factorization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910709469675520 |
|---|---|
| author | Guo, Mingzhao Liu, Dongzhu Simeone, Osvaldo Wen, Dingzhu |
| author_facet | Guo, Mingzhao Liu, Dongzhu Simeone, Osvaldo Wen, Dingzhu |
| contents | This paper presents a novel gradient compression method for federated learning (FL) in wireless systems. The proposed method centers on a low-rank matrix factorization strategy for local gradient compression that is based on one iteration of a distributed Jacobi successive convex approximation (SCA) at each FL round. The low-rank approximation obtained at one round is used as a "warm start" initialization for Jacobi SCA in the next FL round. A new protocol termed over-the-air low-rank compression (Ota-LC) incorporating this gradient compression method with over-the-air computation and error feedback is shown to have lower computation cost and lower communication overhead, while guaranteeing the same inference performance, as compared with existing benchmarks. As an example, when targeting a test accuracy of 70% on the Cifar-10 dataset, Ota-LC reduces total communication costs by at least 33% compared to benchmark schemes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_07496 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Efficient Wireless Federated Learning via Low-Rank Gradient Factorization Guo, Mingzhao Liu, Dongzhu Simeone, Osvaldo Wen, Dingzhu Information Theory Machine Learning Signal Processing This paper presents a novel gradient compression method for federated learning (FL) in wireless systems. The proposed method centers on a low-rank matrix factorization strategy for local gradient compression that is based on one iteration of a distributed Jacobi successive convex approximation (SCA) at each FL round. The low-rank approximation obtained at one round is used as a "warm start" initialization for Jacobi SCA in the next FL round. A new protocol termed over-the-air low-rank compression (Ota-LC) incorporating this gradient compression method with over-the-air computation and error feedback is shown to have lower computation cost and lower communication overhead, while guaranteeing the same inference performance, as compared with existing benchmarks. As an example, when targeting a test accuracy of 70% on the Cifar-10 dataset, Ota-LC reduces total communication costs by at least 33% compared to benchmark schemes. |
| title | Efficient Wireless Federated Learning via Low-Rank Gradient Factorization |
| topic | Information Theory Machine Learning Signal Processing |
| url | https://arxiv.org/abs/2401.07496 |