Quantized Decentralized Stochastic Learning over Directed Graphs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2020
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913620054507520 |
|---|---|
| author | Taheri, Hossein Mokhtari, Aryan Hassani, Hamed Pedarsani, Ramtin |
| author_facet | Taheri, Hossein Mokhtari, Aryan Hassani, Hamed Pedarsani, Ramtin |
| contents | We consider a decentralized stochastic learning problem where data points are distributed among computing nodes communicating over a directed graph. As the model size gets large, decentralized learning faces a major bottleneck that is the heavy communication load due to each node transmitting large messages (model updates) to its neighbors. To tackle this bottleneck, we propose the quantized decentralized stochastic learning algorithm over directed graphs that is based on the push-sum algorithm in decentralized consensus optimization. More importantly, we prove that our algorithm achieves the same convergence rates of the decentralized stochastic learning algorithm with exact-communication for both convex and non-convex losses. Numerical evaluations corroborate our main theoretical results and illustrate significant speed-up compared to the exact-communication methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2002_09964 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | Quantized Decentralized Stochastic Learning over Directed Graphs Taheri, Hossein Mokhtari, Aryan Hassani, Hamed Pedarsani, Ramtin Distributed, Parallel, and Cluster Computing Machine Learning Multiagent Systems Systems and Control Signal Processing We consider a decentralized stochastic learning problem where data points are distributed among computing nodes communicating over a directed graph. As the model size gets large, decentralized learning faces a major bottleneck that is the heavy communication load due to each node transmitting large messages (model updates) to its neighbors. To tackle this bottleneck, we propose the quantized decentralized stochastic learning algorithm over directed graphs that is based on the push-sum algorithm in decentralized consensus optimization. More importantly, we prove that our algorithm achieves the same convergence rates of the decentralized stochastic learning algorithm with exact-communication for both convex and non-convex losses. Numerical evaluations corroborate our main theoretical results and illustrate significant speed-up compared to the exact-communication methods. |
| title | Quantized Decentralized Stochastic Learning over Directed Graphs |
| topic | Distributed, Parallel, and Cluster Computing Machine Learning Multiagent Systems Systems and Control Signal Processing |
| url | https://arxiv.org/abs/2002.09964 |