Asynchronous Distributed Learning with Quantized Finite-Time Coordination
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908515846586368 |
|---|---|
| author | Bastianello, Nicola Rikos, Apostolos I. Johansson, Karl H. |
| author_facet | Bastianello, Nicola Rikos, Apostolos I. Johansson, Karl H. |
| contents | In this paper we address distributed learning problems over peer-to-peer networks. In particular, we focus on the challenges of quantized communications, asynchrony, and stochastic gradients that arise in this set-up. We first discuss how to turn the presence of quantized communications into an advantage, by resorting to a finite-time, quantized coordination scheme. This scheme is combined with a distributed gradient descent method to derive the proposed algorithm. Secondly, we show how this algorithm can be adapted to allow asynchronous operations of the agents, as well as the use of stochastic gradients. Finally, we propose a variant of the algorithm which employs zooming-in quantization. We analyze the convergence of the proposed methods and compare them to state-of-the-art alternatives. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_17156 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Asynchronous Distributed Learning with Quantized Finite-Time Coordination Bastianello, Nicola Rikos, Apostolos I. Johansson, Karl H. Optimization and Control Systems and Control In this paper we address distributed learning problems over peer-to-peer networks. In particular, we focus on the challenges of quantized communications, asynchrony, and stochastic gradients that arise in this set-up. We first discuss how to turn the presence of quantized communications into an advantage, by resorting to a finite-time, quantized coordination scheme. This scheme is combined with a distributed gradient descent method to derive the proposed algorithm. Secondly, we show how this algorithm can be adapted to allow asynchronous operations of the agents, as well as the use of stochastic gradients. Finally, we propose a variant of the algorithm which employs zooming-in quantization. We analyze the convergence of the proposed methods and compare them to state-of-the-art alternatives. |
| title | Asynchronous Distributed Learning with Quantized Finite-Time Coordination |
| topic | Optimization and Control Systems and Control |
| url | https://arxiv.org/abs/2408.17156 |