Energy-efficient Decentralized Learning via Graph Sparsification
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909208659623936 |
|---|---|
| author | Zhang, Xusheng Chiu, Cho-Chun He, Ting |
| author_facet | Zhang, Xusheng Chiu, Cho-Chun He, Ting |
| contents | This work aims at improving the energy efficiency of decentralized learning by optimizing the mixing matrix, which controls the communication demands during the learning process. Through rigorous analysis based on a state-of-the-art decentralized learning algorithm, the problem is formulated as a bi-level optimization, with the lower level solved by graph sparsification. A solution with guaranteed performance is proposed for the special case of fully-connected base topology and a greedy heuristic is proposed for the general case. Simulations based on real topology and dataset show that the proposed solution can lower the energy consumption at the busiest node by 54%-76% while maintaining the quality of the trained model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_03083 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Energy-efficient Decentralized Learning via Graph Sparsification Zhang, Xusheng Chiu, Cho-Chun He, Ting Machine Learning Distributed, Parallel, and Cluster Computing Optimization and Control This work aims at improving the energy efficiency of decentralized learning by optimizing the mixing matrix, which controls the communication demands during the learning process. Through rigorous analysis based on a state-of-the-art decentralized learning algorithm, the problem is formulated as a bi-level optimization, with the lower level solved by graph sparsification. A solution with guaranteed performance is proposed for the special case of fully-connected base topology and a greedy heuristic is proposed for the general case. Simulations based on real topology and dataset show that the proposed solution can lower the energy consumption at the busiest node by 54%-76% while maintaining the quality of the trained model. |
| title | Energy-efficient Decentralized Learning via Graph Sparsification |
| topic | Machine Learning Distributed, Parallel, and Cluster Computing Optimization and Control |
| url | https://arxiv.org/abs/2401.03083 |